Repository navigation
fix(bigquerydataset): correct FullID and selfLink conversion for project IDs containing a colon - #12383
Conversation
BigQuery returns a dataset's FullID as "projectID:datasetID", and three call sites split it on every colon and required exactly two parts. A project ID can itself contain a colon, so those sites broke for: example.com:my-project domain-scoped (legacy, public universe) my-universe:my-project universe-qualified For both, FullID has two colons. Create failed outright with "expected to be in the format of projectID:datasetID", and the resourceID and selfLink mappings silently produced nothing. Split on the last colon instead, via a shared parseDatasetFullID helper. That is correct for every project ID form, including the plain one, so this is a fix in the public universe too and needs no configuration.
…Trim strings.Trim takes a cutset, not a prefix, so it removed any leading or trailing character appearing anywhere in the URL string: in: https://bigquery.googleapis.com/bigquery/v2/projects/my-project/datasets/my_dataset out: jects/my-project/datasets/my_d The subsequent tokens[0] == "projects" guard therefore never matched and BigQueryDatasetStatus_ToProto silently never set FullID. Had it matched, the dataset name was truncated too. This is the inverse of the FullID parsing fixed in the previous commit, so the two directions are now consistent and covered by tests.
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: anfernee The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
Hi, Gently Up. Regards. |
|
There are some test failures, can you take a look @YpNo ? |
Hi @anfernee, |
|
Triggered the rerun! |
|
New changes are detected. LGTM label has been removed. |
|
Hi @anfernee The tests-e2e-fixtures-bigqueryconnection failure is artifact expiry. The job exits in 25s with Artifact not found for name: prebuilt-test-binaries. prepare-test-matrix uploads that with retention-days: 1, and this run was re-run ~26 days after it was created, so the artifact was long gone. I have pushed an empty commit to trigger the whole build CI. |
8914be6
Automated draft of release notes for version 1.158.0 comparing v1.157.0 to v1.158.0. This PR was generated by the overseer,priority/medium,area/release agent (powered by the gemini-3.7-flash model). Fixes GoogleCloudPlatform#13483 ```release-note NONE ``` ## Gathered Release Data # Release Notes Report: `v1.157.0` → `v1.158.0` - **From Tag/Ref:** `v1.157.0` (`e8d55d9935`) - **To Tag/Ref:** `v1.158.0` (`2a495848d1`) - **Total PRs Merged:** 166 - **PRs with Release Notes:** 28 - **PRs with `NONE` Release Note:** 138 ## 🌟 PRs with Release Notes | PR | Title | Author | Release Note | | :--- | :--- | :--- | :--- | | [GoogleCloudPlatform#13418](GoogleCloudPlatform#13418) | Greenfield: Implement MockGCP and Alignment for DataplexAspectType | @hopper-coder-bot | New Alpha Resources (Direct Reconciler): `DataplexAspectType` | | [GoogleCloudPlatform#13429](GoogleCloudPlatform#13429) | Greenfield: Implement MockGCP and Alignment for EventarcMessageBus | @ada-coder-bot | New Alpha Resources (Direct Reconciler): `EventarcMessageBus` | | [GoogleCloudPlatform#13363](GoogleCloudPlatform#13363) | RedisCluster: Add support for managedBackupSource field | @ada-coder-bot | RedisCluster: Add support for managedBackupSource field. | | [GoogleCloudPlatform#12795](GoogleCloudPlatform#12795) | fix(composer): fix false drift and update loops for ComposerEnvironment | @eugenenuke | Fix infinite reconciliation loops, false drift, and HTTP 400 errors on ComposerEnvironment when using empty map fields (`envVariables: {}`, `labels: {}`), partial `workloadsConfig` blocks, or disabling `scheduledSnapshotsConfig`. | | [GoogleCloudPlatform#13353](GoogleCloudPlatform#13353) | Greenfield: Implement MockGCP and Alignment for AgentRegistryBinding | @hopper-coder-bot | New Alpha Resources (Direct Reconciler): `AgentRegistryBinding` | | [GoogleCloudPlatform#12969](GoogleCloudPlatform#12969) | feat(backupdr): Add Cloud SQL Instance support to BackupDRBackupPlanAssociation | @ada-coder-bot | `BackupDRBackupPlanAssociation`: Add `sqlInstanceRef` support to allow backing up Cloud SQL instances. | | [GoogleCloudPlatform#13342](GoogleCloudPlatform#13342) | RedisCluster: Add support for gcsSource field | @neumann-coder-bot | RedisCluster: Add support for gcsSource field. | | [GoogleCloudPlatform#13224](GoogleCloudPlatform#13224) | Greenfield: Implement MockGCP and Alignment for AIPlatformPersistentResource | @neumann-coder-bot | New Alpha Resources (Direct Reconciler): `AIPlatformPersistentResource` | | [GoogleCloudPlatform#13198](GoogleCloudPlatform#13198) | Support Org and Folder level ModelArmorFloorSetting | @hopper-coder-bot | Add Organization and Folder level support to the ModelArmorFloorSetting direct controller. | | [GoogleCloudPlatform#12731](GoogleCloudPlatform#12731) | Support Pause at resource-level | @lovelace-coder-bot | Support pausing and unpausing actuation at the individual resource-level using the annotation `cnrm.cloud.google.com/actuation-mode`. | | [GoogleCloudPlatform#13058](GoogleCloudPlatform#13058) | Greenfield: Implement MockGCP and Alignment for CloudNumberRegistryRegistryBook | @lovelace-coder-bot | New Alpha Resources (Direct Reconciler): `CloudNumberRegistryRegistryBook` | | [GoogleCloudPlatform#11784](GoogleCloudPlatform#11784) | ComputeNetworkFirewallPolicyRule: add support for srcNetworkContext and destNetworkContext fields | @katrielt | Added support for `srcNetworkContext` and `destNetworkContext` in `ComputeNetworkFirewallPolicyRule` (`match` block). | | [GoogleCloudPlatform#12774](GoogleCloudPlatform#12774) | Greenfield: Implement types and identity for NetworkSecurityMirroringDeploymentGroup | @neumann-coder-bot | * New resource `NetworkSecurityMirroringDeploymentGroup` is now available. | | [GoogleCloudPlatform#13099](GoogleCloudPlatform#13099) | Direct controller structured reporting (Diff.Object): computehttpshealthcheck_controller.go | @neumann-coder-bot | Fix CLI preview resource association for ComputeHTTPSHealthCheck by ensuring Diff.Object is populated before structured reporting. | | [GoogleCloudPlatform#13077](GoogleCloudPlatform#13077) | Direct controller structured reporting: Populate Diff.Object in grafeasnote_controller.go | @hopper-coder-bot | Ensure `Diff.Object` is populated before calling `structuredreporting.ReportDiff` in `grafeasnote_controller.go`. | | [GoogleCloudPlatform#13122](GoogleCloudPlatform#13122) | Direct controller structured reporting: populate Diff.Object in pubsubtopic_controller.go | @neumann-coder-bot | Ensure `Diff.Object` is populated with `updateOp.GetUnstructured()` before calling `structuredreporting.ReportDiff` in `pubsubtopic_controller.go`. | | [GoogleCloudPlatform#13144](GoogleCloudPlatform#13144) | Direct controller structured reporting: populate Diff.Object for DatabaseMigrationConversionWorkspace | @neumann-coder-bot | Ensure `Diff.Object` is populated for structured reporting in DatabaseMigrationConversionWorkspace direct controller. | | [GoogleCloudPlatform#13141](GoogleCloudPlatform#13141) | fix(direct): populate Diff.Object in Dialogflow SipTrunk controller | @hopper-coder-bot | fix(direct): populate Diff.Object in Dialogflow SipTrunk controller | | [GoogleCloudPlatform#12903](GoogleCloudPlatform#12903) | Greenfield: Implement direct controller, E2E fixtures, and fuzzer for ModelArmorFloorSetting | @lovelace-coder-bot | Added direct controller, E2E fixtures, and fuzzer for ModelArmorFloorSetting. | | [GoogleCloudPlatform#12896](GoogleCloudPlatform#12896) | Greenfield: Implement MockGCP and Alignment for AssuredWorkloadsWorkload | @lovelace-coder-bot | New Alpha Resources (Direct Reconciler): `AssuredWorkloadsWorkload` | | [GoogleCloudPlatform#12815](GoogleCloudPlatform#12815) | Greenfield: Implement MockGCP and Alignment for VMMigrationGroup | @lovelace-coder-bot | New Alpha Resources (Direct Reconciler): `VMMigrationGroup` | | [GoogleCloudPlatform#12627](GoogleCloudPlatform#12627) | Remove empty status.observedState from v1alpha1-only CRDs | @neumann-coder-bot | fix: Remove empty `status.observedState` from v1alpha1 CRDs. This is to align [CRD design best practices](https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#specifying-a-structural-schema). | | [GoogleCloudPlatform#12813](GoogleCloudPlatform#12813) | ComputeRouterNAT: Support both short name and canonical long path in spec.routerRef | @neumann-coder-bot | ComputeRouterNAT: Support both short name and canonical long path in spec.routerRef in direct controller | | [GoogleCloudPlatform#12763](GoogleCloudPlatform#12763) | Expose spec.settings.databaseReplicationEnabled in SQLInstance | @neumann-coder-bot | Support `spec.settings.databaseReplicationEnabled` in SQLInstance | | [GoogleCloudPlatform#12800](GoogleCloudPlatform#12800) | Implement MockGCP and Alignment for ContactCenterInsightsConversation | @ada-coder-bot | New Alpha Resources (Direct Reconciler): `ContactCenterInsightsConversation` | | [GoogleCloudPlatform#12818](GoogleCloudPlatform#12818) | feat: Support newer networkEndpointType in ComputeRegionNetworkEndpointGroup | @gemmahou | Add "INTERNET_IP_PORT", "INTERNET_FQDN_PORT", "GCE_VM_IP_PORTMAP" value for `networkEndpointType` in the ComputeRegionNetworkEndpointGroup resource | | [GoogleCloudPlatform#12383](GoogleCloudPlatform#12383) | fix(bigquerydataset): correct FullID and selfLink conversion for project IDs containing a colon | @YpNo | Fixed BigQueryDataset failing to reconcile in projects whose project ID contains a colon, such as domain-scoped projects (`example.com:my-project`). Creation previously failed with "The full ID of the created BigQueryDataset is expected to be in the format of projectID:datasetID", and the `resourceID` and `selfLink` fields were not populated. | | [GoogleCloudPlatform#12821](GoogleCloudPlatform#12821) | Greenfield: Align and verify MockGCP for NetworkSecurityInterceptDeployment | @hopper-coder-bot | New Alpha Resources (Direct Reconciler): `NetworkSecurityInterceptDeployment` | ## 📋 Complete PR & Release Note List | PR | Title | Author | Release Note | | :--- | :--- | :--- | :--- | | [GoogleCloudPlatform#13334](GoogleCloudPlatform#13334) | Release 1.158.0 | @ada-coder-bot | NONE | | [GoogleCloudPlatform#13205](GoogleCloudPlatform#13205) | Design: Standardizing resource kind names | @gemmahou | NONE | | [GoogleCloudPlatform#13304](GoogleCloudPlatform#13304) | Greenfield: Implement direct KRM types, identity, and generate.sh for MapManagementStyleConfig | @hopper-coder-bot | NONE | | [GoogleCloudPlatform#13332](GoogleCloudPlatform#13332) | Greenfield: Implement direct controller, E2E fixtures, and fuzzer for NetworkSecurityAddressGroup | @lovelace-coder-bot | NONE | | [GoogleCloudPlatform#13422](GoogleCloudPlatform#13422) | Greenfield: Implement direct controller, E2E fixtures, and fuzzer for BigtableSchemaBundle | @neumann-coder-bot | NONE | | [GoogleCloudPlatform#13439](GoogleCloudPlatform#13439) | chore(workflows): ensure reused GitHub issues have required step labels in kcc-greenfield | @sdowell | NONE | | [GoogleCloudPlatform#13438](GoogleCloudPlatform#13438) | Greenfield: Align MockGCP logs with RealGCP for DataplexAspectType | @neumann-coder-bot | NONE | | [GoogleCloudPlatform#13428](GoogleCloudPlatform#13428) | Greenfield: Align MockGCP logs with RealGCP for DataplexDataTaxonomy | @lovelace-coder-bot | NONE | | [GoogleCloudPlatform#13447](GoogleCloudPlatform#13447) | chore: ignore requests.log | @acpana | NONE | | [GoogleCloudPlatform#13288](GoogleCloudPlatform#13288) | fix(datamigration): migrate DatabaseMigrationMigrationJob.spec.cmekKeyNameRef to kmsv1beta1.KMSCryptoKeyRef | @anhdle-sso | NONE | | [GoogleCloudPlatform#13286](GoogleCloudPlatform#13286) | fix(batch): migrate BatchJob kmsKeyRef to kmsv1beta1.KMSCryptoKeyRef | @anhdle-sso | NONE | | [GoogleCloudPlatform#13287](GoogleCloudPlatform#13287) | fix(compute): migrate ComputeDisk kmsKeyRef fields to kmsv1beta1.KMSCryptoKeyRef | @anhdle-sso | NONE | | [GoogleCloudPlatform#13285](GoogleCloudPlatform#13285) | fix(apigeeregistry): migrate ApigeeRegistryInstance.spec.config.cmekKeyNameRef to kmsv1beta1.KMSCryptoKeyRef | @anhdle-sso | NONE | | [GoogleCloudPlatform#13396](GoogleCloudPlatform#13396) | apichecks: extract reference rules from crds_test.go into controllerbuilder/pkg/refs | @ldanielmadariaga | NONE | | [GoogleCloudPlatform#13431](GoogleCloudPlatform#13431) | chore: Correct cp_resources_list to match the spreadsheet | @gemmahou | NONE | | [GoogleCloudPlatform#12863](GoogleCloudPlatform#12863) | Greenfield: Implement direct controller, E2E fixtures, and fuzzer for CloudBuildConnection | @hopper-coder-bot | NONE | | [GoogleCloudPlatform#13418](GoogleCloudPlatform#13418) | Greenfield: Implement MockGCP and Alignment for DataplexAspectType | @hopper-coder-bot | New Alpha Resources (Direct Reconciler): `DataplexAspectType` | | [GoogleCloudPlatform#13395](GoogleCloudPlatform#13395) | generate-types-and-mappers: always rebuild controllerbuilder | @ldanielmadariaga | NONE | | [GoogleCloudPlatform#13393](GoogleCloudPlatform#13393) | controllerbuilder: drop imports orphaned by --prune-unused-types | @ldanielmadariaga | NONE | | [GoogleCloudPlatform#13392](GoogleCloudPlatform#13392) | apichecks: catch identity collection segments GCP has never seen | @ldanielmadariaga | NONE | | [GoogleCloudPlatform#13429](GoogleCloudPlatform#13429) | Greenfield: Implement MockGCP and Alignment for EventarcMessageBus | @ada-coder-bot | New Alpha Resources (Direct Reconciler): `EventarcMessageBus` | | [GoogleCloudPlatform#13312](GoogleCloudPlatform#13312) | Greenfield: Implement direct KRM types, identity, and generate.sh for NetworkConnectivityTransport | @lovelace-coder-bot | NONE | | [GoogleCloudPlatform#13329](GoogleCloudPlatform#13329) | Add clusterEndpoints to RedisCluster spec and status | @lovelace-coder-bot | NONE | | [GoogleCloudPlatform#13363](GoogleCloudPlatform#13363) | RedisCluster: Add support for managedBackupSource field | @ada-coder-bot | RedisCluster: Add support for managedBackupSource field. | | [GoogleCloudPlatform#13375](GoogleCloudPlatform#13375) | Greenfield: Implement direct controller, E2E fixtures, and fuzzer for EventarcMessageBus | @hopper-coder-bot | NONE | | [GoogleCloudPlatform#13359](GoogleCloudPlatform#13359) | Greenfield: Implement direct KRM types, identity, and generate.sh for BigtableSchemaBundle | @neumann-coder-bot | NONE | | [GoogleCloudPlatform#12630](GoogleCloudPlatform#12630) | mockgcp/dataproc: align DataprocSessionTemplate mock error with real GCP | @hopper-coder-bot | NONE | | [GoogleCloudPlatform#13391](GoogleCloudPlatform#13391) | Release Notes 1.157.0 | @anhdle-sso | NONE | | [GoogleCloudPlatform#13200](GoogleCloudPlatform#13200) | Greenfield: Implement direct controller, E2E fixtures, and fuzzer for DataplexAspectType | @ada-coder-bot | NONE | | [GoogleCloudPlatform#13182](GoogleCloudPlatform#13182) | Greenfield: Implement direct KRM types, identity, and generate.sh for DataLineageProcess | @hopper-coder-bot | NONE | | [GoogleCloudPlatform#12795](GoogleCloudPlatform#12795) | fix(composer): fix false drift and update loops for ComposerEnvironment | @eugenenuke | Fix infinite reconciliation loops, false drift, and HTTP 400 errors on ComposerEnvironment when using empty map fields (`envVariables: {}`, `labels: {}`), partial `workloadsConfig` blocks, or disabling `scheduledSnapshotsConfig`. | | [GoogleCloudPlatform#13328](GoogleCloudPlatform#13328) | Greenfield: Implement direct KRM types, identity, and generate.sh for EventarcMessageBus | @lovelace-coder-bot | NONE | | [GoogleCloudPlatform#13368](GoogleCloudPlatform#13368) | Chore: Corrected GCP Resources CRUD operations | @gemmahou | NONE | | [GoogleCloudPlatform#13339](GoogleCloudPlatform#13339) | Greenfield: Implement direct KRM types, identity, and generate.sh for OracleDatabaseExadbVMCluster | @neumann-coder-bot | NONE | | [GoogleCloudPlatform#13305](GoogleCloudPlatform#13305) | Greenfield: Implement direct controller, E2E fixtures, and fuzzer for DiscoveryEngineSampleQuerySet | @hopper-coder-bot | NONE | | [GoogleCloudPlatform#13314](GoogleCloudPlatform#13314) | Greenfield: Implement direct KRM types, identity, and generate.sh for NetworkServicesLBEdgeExtension | @ada-coder-bot | NONE | | [GoogleCloudPlatform#13353](GoogleCloudPlatform#13353) | Greenfield: Implement MockGCP and Alignment for AgentRegistryBinding | @hopper-coder-bot | New Alpha Resources (Direct Reconciler): `AgentRegistryBinding` | | [GoogleCloudPlatform#13366](GoogleCloudPlatform#13366) | test(compute): add unit tests for compareComputeRouterNAT to verify migration diff discrepancies | @neumann-coder-bot | NONE | | [GoogleCloudPlatform#13361](GoogleCloudPlatform#13361) | Greenfield: Align MockGCP logs with RealGCP for AIPlatformPersistentResource | @hopper-coder-bot | NONE | | [GoogleCloudPlatform#12969](GoogleCloudPlatform#12969) | feat(backupdr): Add Cloud SQL Instance support to BackupDRBackupPlanAssociation | @ada-coder-bot | `BackupDRBackupPlanAssociation`: Add `sqlInstanceRef` support to allow backing up Cloud SQL instances. | | [GoogleCloudPlatform#13362](GoogleCloudPlatform#13362) | chore(apichecks): update missingfields.txt for RedisCluster gcsSource.uris | @anhdle-sso | NONE | | [GoogleCloudPlatform#13342](GoogleCloudPlatform#13342) | RedisCluster: Add support for gcsSource field | @neumann-coder-bot | RedisCluster: Add support for gcsSource field. | | [GoogleCloudPlatform#13238](GoogleCloudPlatform#13238) | fix(kms): read status.selfLink in KMSCryptoKeyRef.Normalize fallback | @anhdle-sso | NONE | | [GoogleCloudPlatform#13245](GoogleCloudPlatform#13245) | fix(artifactregistry): migrate ArtifactRegistryRepository.spec.kmsKeyRef to kmsv1beta1.KMSCryptoKeyRef | @anhdle-sso | NONE | | [GoogleCloudPlatform#13234](GoogleCloudPlatform#13234) | chore: sync workflows from overseer repo | @sdowell | NONE | | [GoogleCloudPlatform#12776](GoogleCloudPlatform#12776) | Add more details into the boolean handling in TF doc | @maqiuyujoyce | NONE | | [GoogleCloudPlatform#13224](GoogleCloudPlatform#13224) | Greenfield: Implement MockGCP and Alignment for AIPlatformPersistentResource | @neumann-coder-bot | New Alpha Resources (Direct Reconciler): `AIPlatformPersistentResource` | | [GoogleCloudPlatform#13225](GoogleCloudPlatform#13225) | Greenfield: Implement direct controller, E2E fixtures, and fuzzer for AgentRegistryBinding | @hopper-coder-bot | NONE | | [GoogleCloudPlatform#13198](GoogleCloudPlatform#13198) | Support Org and Folder level ModelArmorFloorSetting | @hopper-coder-bot | Add Organization and Folder level support to the ModelArmorFloorSetting direct controller. | | [GoogleCloudPlatform#12731](GoogleCloudPlatform#12731) | Support Pause at resource-level | @lovelace-coder-bot | Support pausing and unpausing actuation at the individual resource-level using the annotation `cnrm.cloud.google.com/actuation-mode`. | | [GoogleCloudPlatform#12936](GoogleCloudPlatform#12936) | Create the record-real-gcp skill | @maqiuyujoyce | NONE | | [GoogleCloudPlatform#13211](GoogleCloudPlatform#13211) | Greenfield: Implement direct controller, E2E fixtures, and fuzzer for DataplexDataTaxonomy | @ada-coder-bot | NONE | | [GoogleCloudPlatform#11386](GoogleCloudPlatform#11386) | ai:chore: Implement direct types for: AgentRegistryBinding | @ada-coder-bot | NONE | | [GoogleCloudPlatform#13201](GoogleCloudPlatform#13201) | Greenfield: Implement direct controller, E2E fixtures, and fuzzer for AIPlatformPersistentResource | @lovelace-coder-bot | NONE | | [GoogleCloudPlatform#13058](GoogleCloudPlatform#13058) | Greenfield: Implement MockGCP and Alignment for CloudNumberRegistryRegistryBook | @lovelace-coder-bot | New Alpha Resources (Direct Reconciler): `CloudNumberRegistryRegistryBook` | | [GoogleCloudPlatform#12870](GoogleCloudPlatform#12870) | Greenfield: Implement types, identity, and generate.sh for CloudNumberRegistryIpamAdminScope | @ada-coder-bot | NONE | | [GoogleCloudPlatform#11784](GoogleCloudPlatform#11784) | ComputeNetworkFirewallPolicyRule: add support for srcNetworkContext and destNetworkContext fields | @katrielt | Added support for `srcNetworkContext` and `destNetworkContext` in `ComputeNetworkFirewallPolicyRule` (`match` block). | | [GoogleCloudPlatform#12774](GoogleCloudPlatform#12774) | Greenfield: Implement types and identity for NetworkSecurityMirroringDeploymentGroup | @neumann-coder-bot | * New resource `NetworkSecurityMirroringDeploymentGroup` is now available. | | [GoogleCloudPlatform#13121](GoogleCloudPlatform#13121) | Direct controller structured reporting: bigqueryanalyticshubdataexchange_controller.go | @hopper-coder-bot | NONE | | [GoogleCloudPlatform#13099](GoogleCloudPlatform#13099) | Direct controller structured reporting (Diff.Object): computehttpshealthcheck_controller.go | @neumann-coder-bot | Fix CLI preview resource association for ComputeHTTPSHealthCheck by ensuring Diff.Object is populated before structured reporting. | | [GoogleCloudPlatform#13168](GoogleCloudPlatform#13168) | Direct controller structured reporting (Diff.Object): apihubapi_controller.go | @neumann-coder-bot | NONE | | [GoogleCloudPlatform#13171](GoogleCloudPlatform#13171) | Structured reporting: populate Diff.Object in ccinsightsphrasematcher | @ada-coder-bot | NONE | | [GoogleCloudPlatform#13115](GoogleCloudPlatform#13115) | Direct controller structured reporting: kmskeyring_controller.go | @lovelace-coder-bot | NONE | | [GoogleCloudPlatform#13132](GoogleCloudPlatform#13132) | Direct controller structured reporting (Diff.Object): discoveryengineuserstore_controller.go | @neumann-coder-bot | NONE | | [GoogleCloudPlatform#13142](GoogleCloudPlatform#13142) | Direct controller structured reporting: computerouterinterface_controller.go | @neumann-coder-bot | NONE | | [GoogleCloudPlatform#13150](GoogleCloudPlatform#13150) | Direct controller structured reporting (Diff.Object): apihubattribute_controller.go | @ada-coder-bot | NONE | | [GoogleCloudPlatform#13093](GoogleCloudPlatform#13093) | Populate Diff.Object before ReportDiff in computenetwork_controller.go | @hopper-coder-bot | NONE | | [GoogleCloudPlatform#13139](GoogleCloudPlatform#13139) | Direct controller structured reporting (Diff.Object): storageinsightsdatasetconfig_controller.go | @hopper-coder-bot | NONE | | [GoogleCloudPlatform#13077](GoogleCloudPlatform#13077) | Direct controller structured reporting: Populate Diff.Object in grafeasnote_controller.go | @hopper-coder-bot | Ensure `Diff.Object` is populated before calling `structuredreporting.ReportDiff` in `grafeasnote_controller.go`. | | [GoogleCloudPlatform#13081](GoogleCloudPlatform#13081) | Direct controller structured reporting (Diff.Object): notebooksexecution_controller.go | @lovelace-coder-bot | NONE | | [GoogleCloudPlatform#13091](GoogleCloudPlatform#13091) | Structured Reporting: Populate Diff.Object in computerouternat_controller.go | @hopper-coder-bot | NONE | | [GoogleCloudPlatform#13109](GoogleCloudPlatform#13109) | Direct controller structured reporting (Diff.Object): apihubdependency_controller.go | @lovelace-coder-bot | NONE | | [GoogleCloudPlatform#13122](GoogleCloudPlatform#13122) | Direct controller structured reporting: populate Diff.Object in pubsubtopic_controller.go | @neumann-coder-bot | Ensure `Diff.Object` is populated with `updateOp.GetUnstructured()` before calling `structuredreporting.ReportDiff` in `pubsubtopic_controller.go`. | | [GoogleCloudPlatform#13124](GoogleCloudPlatform#13124) | Direct controller structured reporting (Diff.Object): securesourcemanagerinstance_controller.go | @neumann-coder-bot | NONE | | [GoogleCloudPlatform#13131](GoogleCloudPlatform#13131) | Direct: Populate report.Object in speechcustomclass_controller | @hopper-coder-bot | NONE | | [GoogleCloudPlatform#13144](GoogleCloudPlatform#13144) | Direct controller structured reporting: populate Diff.Object for DatabaseMigrationConversionWorkspace | @neumann-coder-bot | Ensure `Diff.Object` is populated for structured reporting in DatabaseMigrationConversionWorkspace direct controller. | | [GoogleCloudPlatform#13148](GoogleCloudPlatform#13148) | Direct controller structured reporting (Diff.Object): computenetworkattachment_controller.go | @neumann-coder-bot | NONE | | [GoogleCloudPlatform#13149](GoogleCloudPlatform#13149) | Direct controller structured reporting (Diff.Object): assuredworkloadsworkload_controller.go | @neumann-coder-bot | NONE | | [GoogleCloudPlatform#13163](GoogleCloudPlatform#13163) | Direct controller structured reporting (Diff.Object): discoveryenginelicenseconfig_controller.go | @ada-coder-bot | NONE | | [GoogleCloudPlatform#13157](GoogleCloudPlatform#13157) | Direct controller structured reporting: computefuturereservation_controller.go | @ada-coder-bot | NONE | | [GoogleCloudPlatform#13155](GoogleCloudPlatform#13155) | Fix ReportDiff usage in NetworkServicesAuthzExtension controller | @neumann-coder-bot | NONE | | [GoogleCloudPlatform#13162](GoogleCloudPlatform#13162) | Direct controller structured reporting (Diff.Object): dialogflowgenerator_controller.go | @hopper-coder-bot | NONE | | [GoogleCloudPlatform#13160](GoogleCloudPlatform#13160) | Direct controller structured reporting (Diff.Object): pubsubsubscription_controller.go | @neumann-coder-bot | NONE | | [GoogleCloudPlatform#13102](GoogleCloudPlatform#13102) | Direct controller structured reporting (Diff.Object): computerouter_controller.go | @neumann-coder-bot | NONE | | [GoogleCloudPlatform#13106](GoogleCloudPlatform#13106) | Direct controller structured reporting (Diff.Object): apigeeregistryinstance_controller.go | @neumann-coder-bot | NONE | | [GoogleCloudPlatform#13111](GoogleCloudPlatform#13111) | Direct controller structured reporting (Diff.Object): aiplatformmodel_controller.go | @neumann-coder-bot | NONE | | [GoogleCloudPlatform#13114](GoogleCloudPlatform#13114) | Direct controller structured reporting: apihubplugin_controller.go | @hopper-coder-bot | NONE | | [GoogleCloudPlatform#13117](GoogleCloudPlatform#13117) | direct/apihub: populate Diff.Object in apihubexternalapi controller | @neumann-coder-bot | NONE | | [GoogleCloudPlatform#13110](GoogleCloudPlatform#13110) | Direct: Populate Diff.Object in DatabaseMigrationMigrationJob controller | @neumann-coder-bot | NONE | | [GoogleCloudPlatform#13116](GoogleCloudPlatform#13116) | Direct controller structured reporting (Diff.Object): computeinstancegroup_controller.go | @lovelace-coder-bot | NONE | | [GoogleCloudPlatform#13113](GoogleCloudPlatform#13113) | Ensure Diff.Object is populated in vertexaipipelinejob_controller.go | @neumann-coder-bot | NONE | | [GoogleCloudPlatform#13118](GoogleCloudPlatform#13118) | Direct controller structured reporting: monitoringalertpolicy_controller.go | @hopper-coder-bot | NONE | | [GoogleCloudPlatform#13119](GoogleCloudPlatform#13119) | Direct controller structured reporting: memorystoreinstanceendpoint_controller.go | @neumann-coder-bot | NONE | | [GoogleCloudPlatform#13120](GoogleCloudPlatform#13120) | Direct controller structured reporting: dataformteamfolder_controller.go | @hopper-coder-bot | NONE | | [GoogleCloudPlatform#13080](GoogleCloudPlatform#13080) | Direct controller structured reporting (Diff.Object): networkmanagementvpcflowlogsconfig_controller.go | @ada-coder-bot | NONE | | [GoogleCloudPlatform#13079](GoogleCloudPlatform#13079) | direct: populate Diff.Object for DiscoveryEngineServingConfig | @neumann-coder-bot | NONE | | [GoogleCloudPlatform#13082](GoogleCloudPlatform#13082) | Direct: populate Diff.Object in notebooksschedule_controller | @lovelace-coder-bot | NONE | | [GoogleCloudPlatform#13075](GoogleCloudPlatform#13075) | Direct: Populate Diff.Object in pubsubsnapshot_controller | @hopper-coder-bot | NONE | | [GoogleCloudPlatform#13094](GoogleCloudPlatform#13094) | Direct: Populate Diff.Object in devconnectinsightsconfig_controller.go | @hopper-coder-bot | NONE | | [GoogleCloudPlatform#13092](GoogleCloudPlatform#13092) | Direct controller structured reporting (Diff.Object): computetargethttpsproxy_controller.go | @neumann-coder-bot | NONE | | [GoogleCloudPlatform#13067](GoogleCloudPlatform#13067) | Direct: Populate Diff.Object in notebooksenvironment_controller.go | @hopper-coder-bot | NONE | | [GoogleCloudPlatform#13072](GoogleCloudPlatform#13072) | Direct controller structured reporting (Diff.Object): eventarcgoogleapisource_controller.go | @hopper-coder-bot | NONE | | [GoogleCloudPlatform#13159](GoogleCloudPlatform#13159) | Direct controller structured reporting: redisclusterendpoint_controller.go | @lovelace-coder-bot | NONE | | [GoogleCloudPlatform#13125](GoogleCloudPlatform#13125) | Direct controller structured reporting (Diff.Object): notebookinstancev2_controller.go | @hopper-coder-bot | NONE | | [GoogleCloudPlatform#13071](GoogleCloudPlatform#13071) | Direct controller structured reporting (Diff.Object): tagstagbinding_controller.go | @neumann-coder-bot | NONE | | [GoogleCloudPlatform#13100](GoogleCloudPlatform#13100) | Direct controller structured reporting (Diff.Object): computeautoscaler_controller.go | @ada-coder-bot | NONE | | [GoogleCloudPlatform#13074](GoogleCloudPlatform#13074) | Direct controller structured reporting (Diff.Object): networkservicesgateway_controller.go | @ada-coder-bot | NONE | | [GoogleCloudPlatform#13069](GoogleCloudPlatform#13069) | Direct: populate Diff.Object in tagslocationtagbinding_controller.go | @neumann-coder-bot | NONE | | [GoogleCloudPlatform#13096](GoogleCloudPlatform#13096) | Direct controller structured reporting (Diff.Object): computeimage_controller.go | @ada-coder-bot | NONE | | [GoogleCloudPlatform#13153](GoogleCloudPlatform#13153) | Direct controller structured reporting (Diff.Object): modelarmortemplate_controller.go | @ada-coder-bot | NONE | | [GoogleCloudPlatform#13112](GoogleCloudPlatform#13112) | Direct controller structured reporting (Diff.Object): computefirewall_controller.go | @lovelace-coder-bot | NONE | | [GoogleCloudPlatform#13141](GoogleCloudPlatform#13141) | fix(direct): populate Diff.Object in Dialogflow SipTrunk controller | @hopper-coder-bot | fix(direct): populate Diff.Object in Dialogflow SipTrunk controller | | [GoogleCloudPlatform#13076](GoogleCloudPlatform#13076) | Direct: Populate Diff.Object in networkserviceshttproute_controller.go | @ada-coder-bot | NONE | | [GoogleCloudPlatform#13164](GoogleCloudPlatform#13164) | Direct controller structured reporting (Diff.Object): ccinsightsqascorecard_controller.go | @hopper-coder-bot | NONE | | [GoogleCloudPlatform#13152](GoogleCloudPlatform#13152) | Direct controller structured reporting: securitycentermanagement | @neumann-coder-bot | NONE | | [GoogleCloudPlatform#13158](GoogleCloudPlatform#13158) | apihub: populate Diff.Object in apihubdeployment_controller.go | @ada-coder-bot | NONE | | [GoogleCloudPlatform#13161](GoogleCloudPlatform#13161) | Direct controller structured reporting (Diff.Object): apihubcuration_controller.go | @ada-coder-bot | NONE | | [GoogleCloudPlatform#13129](GoogleCloudPlatform#13129) | Direct controller structured reporting (Diff.Object): memorystoreinstance_controller.go | @lovelace-coder-bot | NONE | | [GoogleCloudPlatform#13127](GoogleCloudPlatform#13127) | firestore: populate Diff.Object in firestoredatabase_controller.go | @ada-coder-bot | NONE | | [GoogleCloudPlatform#13101](GoogleCloudPlatform#13101) | Direct: Populate Diff.Object in bigtablegcpolicy_controller.go | @lovelace-coder-bot | NONE | | [GoogleCloudPlatform#13126](GoogleCloudPlatform#13126) | Direct controller structured reporting (Diff.Object): tagstagvalue_controller.go | @lovelace-coder-bot | NONE | | [GoogleCloudPlatform#13063](GoogleCloudPlatform#13063) | direct: populate Diff.Object in PubSubSchema controller | @ada-coder-bot | NONE | | [GoogleCloudPlatform#13086](GoogleCloudPlatform#13086) | KMS: Populate Diff.Object in KMSCryptoKey controller | @lovelace-coder-bot | NONE | | [GoogleCloudPlatform#13098](GoogleCloudPlatform#13098) | compute: populate Diff.Object in ComputeFirewallPolicy controller | @lovelace-coder-bot | NONE | | [GoogleCloudPlatform#13123](GoogleCloudPlatform#13123) | aiplatform: populate Diff.Object in vertexaitrainingpipeline_controller | @lovelace-coder-bot | NONE | | [GoogleCloudPlatform#13138](GoogleCloudPlatform#13138) | direct: populate Diff.Object in devicestreamingsession_controller | @hopper-coder-bot | NONE | | [GoogleCloudPlatform#13095](GoogleCloudPlatform#13095) | Direct controller structured reporting (Diff.Object): computehttphealthcheck_controller | @ada-coder-bot | NONE | | [GoogleCloudPlatform#13066](GoogleCloudPlatform#13066) | Direct controller structured reporting (Diff.Object): notebookinstance_controller.go | @neumann-coder-bot | NONE | | [GoogleCloudPlatform#13085](GoogleCloudPlatform#13085) | Direct controller structured reporting: geminidataanalyticsconversation_controller.go | @lovelace-coder-bot | NONE | | [GoogleCloudPlatform#13064](GoogleCloudPlatform#13064) | Direct controller structured reporting (Diff.Object): tagstagkey_controller.go | @hopper-coder-bot | NONE | | [GoogleCloudPlatform#12862](GoogleCloudPlatform#12862) | Greenfield: Implement direct KRM types, identity, and generate.sh for AIPlatformPersistentResource | @neumann-coder-bot | NONE | | [GoogleCloudPlatform#13084](GoogleCloudPlatform#13084) | Direct controller structured reporting (Diff.Object): kmskeyhandle_controller.go | @neumann-coder-bot | NONE | | [GoogleCloudPlatform#13083](GoogleCloudPlatform#13083) | Direct controller structured reporting (Diff.Object): livestreamasset_controller.go | @ada-coder-bot | NONE | | [GoogleCloudPlatform#12966](GoogleCloudPlatform#12966) | ai:skills: updates | @acpana | NONE | | [GoogleCloudPlatform#12932](GoogleCloudPlatform#12932) | Greenfield: Align MockGCP logs with RealGCP for AssuredWorkloadsWorkload | @ada-coder-bot | NONE | | [GoogleCloudPlatform#12903](GoogleCloudPlatform#12903) | Greenfield: Implement direct controller, E2E fixtures, and fuzzer for ModelArmorFloorSetting | @lovelace-coder-bot | Added direct controller, E2E fixtures, and fuzzer for ModelArmorFloorSetting. | | [GoogleCloudPlatform#12895](GoogleCloudPlatform#12895) | Greenfield: Implement direct controller, E2E fixtures, and fuzzer for CloudNumberRegistryRegistryBook | @ada-coder-bot | NONE | | [GoogleCloudPlatform#12933](GoogleCloudPlatform#12933) | fix(preview): print alternative controller results when controller type differs | @anhdle-sso | NONE | | [GoogleCloudPlatform#12915](GoogleCloudPlatform#12915) | Greenfield: Align MockGCP logs with RealGCP for ContactCenterInsightsConversation | @hopper-coder-bot | NONE | | [GoogleCloudPlatform#12896](GoogleCloudPlatform#12896) | Greenfield: Implement MockGCP and Alignment for AssuredWorkloadsWorkload | @lovelace-coder-bot | New Alpha Resources (Direct Reconciler): `AssuredWorkloadsWorkload` | | [GoogleCloudPlatform#12803](GoogleCloudPlatform#12803) | Greenfield: Align MockGCP logs with RealGCP for APIHubDependency | @lovelace-coder-bot | NONE | | [GoogleCloudPlatform#12914](GoogleCloudPlatform#12914) | Greenfield: Implement direct KRM types, identity, and generate.sh for RunWorkerPool | @hopper-coder-bot | NONE | | [GoogleCloudPlatform#12926](GoogleCloudPlatform#12926) | Greenfield: Align MockGCP logs with RealGCP for VMMigrationGroup | @ada-coder-bot | NONE | | [GoogleCloudPlatform#12921](GoogleCloudPlatform#12921) | feat(common): support optional field exclusion in NormalizeReferences | @neumann-coder-bot | NONE | | [GoogleCloudPlatform#12815](GoogleCloudPlatform#12815) | Greenfield: Implement MockGCP and Alignment for VMMigrationGroup | @lovelace-coder-bot | New Alpha Resources (Direct Reconciler): `VMMigrationGroup` | | [GoogleCloudPlatform#12865](GoogleCloudPlatform#12865) | Greenfield: Implement KRM types, identity, and generate.sh for CloudSupportSupportEventSubscription | @hopper-coder-bot | NONE | | [GoogleCloudPlatform#12900](GoogleCloudPlatform#12900) | refactor(composer): separate rawDesiredPb from mergedDesiredPb to avoid in-place mutation | @hopper-coder-bot | NONE | | [GoogleCloudPlatform#12873](GoogleCloudPlatform#12873) | docs(container): document cpuCfsQuota defaulting behavior in kubeletConfig | @cdmello-g | NONE | | [GoogleCloudPlatform#12627](GoogleCloudPlatform#12627) | Remove empty status.observedState from v1alpha1-only CRDs | @neumann-coder-bot | fix: Remove empty `status.observedState` from v1alpha1 CRDs. This is to align [CRD design best practices](https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#specifying-a-structural-schema). | | [GoogleCloudPlatform#12878](GoogleCloudPlatform#12878) | Improve skills to ensure real GCP logs are recorded for all test cases | @neumann-coder-bot | NONE | | [GoogleCloudPlatform#12813](GoogleCloudPlatform#12813) | ComputeRouterNAT: Support both short name and canonical long path in spec.routerRef | @neumann-coder-bot | ComputeRouterNAT: Support both short name and canonical long path in spec.routerRef in direct controller | | [GoogleCloudPlatform#12763](GoogleCloudPlatform#12763) | Expose spec.settings.databaseReplicationEnabled in SQLInstance | @neumann-coder-bot | Support `spec.settings.databaseReplicationEnabled` in SQLInstance | | [GoogleCloudPlatform#12909](GoogleCloudPlatform#12909) | chore(release): use gather-release-notes in release overseer agent | @anhdle-sso | NONE | | [GoogleCloudPlatform#12800](GoogleCloudPlatform#12800) | Implement MockGCP and Alignment for ContactCenterInsightsConversation | @ada-coder-bot | New Alpha Resources (Direct Reconciler): `ContactCenterInsightsConversation` | | [GoogleCloudPlatform#12880](GoogleCloudPlatform#12880) | refactor(direct): encapsulate label comparison in CompareBrownfieldSpecAndLabels | @hopper-coder-bot | NONE | | [GoogleCloudPlatform#12893](GoogleCloudPlatform#12893) | chore: Revise the Skill for adding fixtures test | @gemmahou | NONE | | [GoogleCloudPlatform#12879](GoogleCloudPlatform#12879) | docs(skills): create skill kcc-direct-brownfield-labels | @ada-coder-bot | NONE | | [GoogleCloudPlatform#12872](GoogleCloudPlatform#12872) | Enrich cp_resources_list.json with protobuf and client references | @anhdle-sso | NONE | | [GoogleCloudPlatform#11566](GoogleCloudPlatform#11566) | Implement direct controller and test fixtures for FilestoreInstance | @lovelace-coder-bot | NONE | | [GoogleCloudPlatform#12818](GoogleCloudPlatform#12818) | feat: Support newer networkEndpointType in ComputeRegionNetworkEndpointGroup | @gemmahou | Add "INTERNET_IP_PORT", "INTERNET_FQDN_PORT", "GCE_VM_IP_PORTMAP" value for `networkEndpointType` in the ComputeRegionNetworkEndpointGroup resource | | [GoogleCloudPlatform#12752](GoogleCloudPlatform#12752) | Greenfield: Implement direct KRM types, identity, and generate.sh for ModelArmorFloorSetting | @hopper-coder-bot | NONE | | [GoogleCloudPlatform#12804](GoogleCloudPlatform#12804) | Greenfield: Implement direct controller, E2E fixtures, and fuzzer for AssuredWorkloadsWorkload | @sdowell | NONE | | [GoogleCloudPlatform#12864](GoogleCloudPlatform#12864) | Greenfield: Implement direct KRM types, identity, and generate.sh for CloudNumberRegistryRegistryBook | @lovelace-coder-bot | NONE | | [GoogleCloudPlatform#12383](GoogleCloudPlatform#12383) | fix(bigquerydataset): correct FullID and selfLink conversion for project IDs containing a colon | @YpNo | Fixed BigQueryDataset failing to reconcile in projects whose project ID contains a colon, such as domain-scoped projects (`example.com:my-project`). Creation previously failed with "The full ID of the created BigQueryDataset is expected to be in the format of projectID:datasetID", and the `resourceID` and `selfLink` fields were not populated. | | [GoogleCloudPlatform#12821](GoogleCloudPlatform#12821) | Greenfield: Align and verify MockGCP for NetworkSecurityInterceptDeployment | @hopper-coder-bot | New Alpha Resources (Direct Reconciler): `NetworkSecurityInterceptDeployment` | | [GoogleCloudPlatform#12876](GoogleCloudPlatform#12876) | refactor(direct): dynamically build parent proto messages in populateDesiredWithActualIfComputed | @lovelace-coder-bot | NONE |
BRIEF Change description
Fixes #12382
BigQueryDatasetconverts between BigQuery'sFullID(projectID:datasetID) and a selfLink URL. Both directions are wrong when the project ID itself contains a colon.Direction 1 :
FullID->resourceID/selfLink. Three call sites splitFullIDon every colon and require exactly two parts. A project ID can contain a colon:FullIDmy-project:my_datasetexample.com:my-project:my_datasetmy-universe:my-project:my_datasetFor the latter two, create fails outright with
The full ID of the created BigQueryDataset is expected to be in the format of projectID:datasetID, and theresourceIDandselfLinkmappings silently produce nothing.Fixed by splitting on the last colon, via a shared
parseDatasetFullIDhelper. Correct for every form, including the plain one.Direction 2 : selfLink ->
FullID.BigQueryDatasetStatus_ToProtostripped the URL prefix withstrings.Trim, which takes a cutset rather than a prefix, so it removed characters from both ends:The following
tokens[0] == "projects"guard therefore never matched, soFullIDwas silently never set — and had it matched, the dataset name was truncated. Changed tostrings.TrimPrefix.WHY do we need this change?
Direction 1 is a user-visible failure: creating a
BigQueryDatasetin a domain-scoped project (example.com:my-project) fails with an error saying the ID is malformed when it is valid. Domain-scoped project IDs have existed in public GCP for years.Direction 2 is a silent correctness bug in the inverse mapping, affecting all projects.
I found these while working on universe support (#12374). Universe-qualified project IDs have the same shape as domain-scoped ones. But neither fix is universe-specific. Both are wrong in the public universe today, and neither needs any configuration.
Special notes for your reviewer:
The two fixes are in separate commits, so the second can be dropped if you would rather it went separately. I kept them together because they are the two directions of the same conversion, in the same file, and fixing only one would leave the pair inconsistent.
strings.Split(selfLink, "/")in direction 2 is left as-is and is correct. Project IDs cannot contain/, so a colon inside one survives the split intact.No API, schema, or behavioural change for project IDs without a colon.
parseDatasetFullIDreturnsok=falsefor malformed input, preserving the existing error path in the controller.Does this PR add something which needs to be 'release noted'?
Additional documentation e.g., references, usage docs, etc.:
Intended Milestone
No preference. Self-contained bug fix, no API change.
Tests you have done
New unit tests:
pkg/controller/direct/bigquerydataset/utils_test.go:parseDatasetFullIDacross plain, domain-scoped and universe-qualified project IDs, plus rejection cases (no colon, empty, empty project, empty dataset, bare colon).pkg/controller/direct/bigquerydataset/bigquerydataset_mappings_test.go:resourceIDandselfLinkderived fromFullID, andFullIDderived back from a selfLink, each across all three project ID forms and a malformed input.make ready-prto ensure this PR is ready for review.