discoveryengine datastore class does not contain billing_estimation attribute

[ffeldhaus]

Determine this is the right repository

• I determined this is the correct repository in which to report this bug.

Summary of the issue

Context

Any method returning a datastore such as list or get as well as initializing a data store object directly returns an object which is missing the billing_estimation attribute.

The REST API documentation clearly shows that billingEstimation is part of the datastore class. Also a REST API call to get a datastore returns the billingEstimation attribute.

Expected Behavior:

The following code (requiring the data store name as data_store_name) should return the billing_estimation

from google.cloud import discoveryengine_v1
client = discoveryengine_v1.DataStoreServiceClient()
data_store = client.get_data_store(name=data_store_name)
data_store.billing_estimation

Actual Behavior:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "venv/lib/python3.12/site-packages/proto/message.py", line 906, in __getattr__
    raise AttributeError(
AttributeError: Unknown field for DataStore: billing_estimation

REST API Behavior:

$ curl -H "Authorization: Bearer $(gcloud auth print-
access-token)" https://discoveryengine.googleapis.com/v1/projects
/$GCP_PROJECT_ID/locations/$LOCATION/collections/default_collection/dataStores/$DATA_STORE_NAME

{
  "name": "<redacted>",
  "displayName": "<redacted>",
  "industryVertical": "GENERIC",
  "createTime": "2024-01-09T18:06:29.019077Z",
  "solutionTypes": [
    "SOLUTION_TYPE_CHAT"
  ],
  "contentConfig": "PUBLIC_WEBSITE",
  "defaultSchemaId": "default_schema",
  "billingEstimation": {
    "websiteDataSize": "45056000",
    "websiteDataUpdateTime": "2024-09-30T11:04:54.236883880Z"
  },
  "servingConfigDataStore": {}
}

API client name and version

google-cloud-discoveryengine 0.12.2

Reproduction steps: code

file: main.py

from google.cloud import discoveryengine_v1
data_store = discoveryengine_v1.DataStore()
data_store.billing_estimation

Reproduction steps: supporting files

No response

Reproduction steps: actual results

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "venv/lib/python3.12/site-packages/proto/message.py", line 906, in __getattr__
    raise AttributeError(
AttributeError: Unknown field for DataStore: billing_estimation

Reproduction steps: expected results

No exception.

OS & version + platform

macOS 14.6.1

Python environment

Python 3.12.6

Python dependencies

Package Version

---

cachetools 5.5.0
certifi 2024.8.30
charset-normalizer 3.3.2
google-api-core 2.20.0
google-auth 2.35.0
google-cloud-discoveryengine 0.12.2
googleapis-common-protos 1.65.0
grpcio 1.66.2
grpcio-status 1.66.2
idna 3.10
pip 24.2
proto-plus 1.24.0
protobuf 5.28.2
pyasn1 0.6.1
pyasn1_modules 0.4.1
requests 2.32.3
rsa 4.9
urllib3 2.2.3

[ohmayr]

@ffeldhaus Thanks for reporting this. I can confirm that the proto definitions used here do not have the billing_estimation field and need to be updated. I've filed an internal issue to update the definitions and will follow up on this.

Meanwhile, can you try using transport="rest" and see if you can access the field by doing the following:

from google.cloud import discoveryengine_v1
client = discoveryengine_v1.DataStoreServiceClient(transport="rest")
data_store = client.get_data_store(name=data_store_name)  # convert the response to a dictionary
response = data_store.json()
billing_estimation = response.get('billing_estimation', None)

I didn't try the above approach but I think you should still be able to access the field in the meanwhile.

[ohmayr]

Googlers see: b/371606109

[ffeldhaus]

Meanwhile, can you try using transport="rest" and see if you can access the field by doing the following:

I tried using transport rest but it has the same issue, billing_estimation also does not exist. I think it is using the same proto definition as the default grpc.

[ohmayr]

You're right. There isn't a defined accessor for the property. What I suggested was based on the assumption that the property should still be accessible somehow given that it's exposed by the server (something I haven't tried). Nonetheless, I'll provide an update once we have the protos updated which should fix the problem.

[parthea]

This was fixed in #13142 which was released in #13143 .
Use google-cloud-discoveryengine: 0.12.3 available here https://pypi.org/project/google-cloud-discoveryengine/0.12.3/

google-cloud-python/packages/google-cloud-discoveryengine/google/cloud/discoveryengine_v1/types/data_store.py

Lines 80 to 82 in 27ea6e2

	billing_estimation (google.cloud.discoveryengine_v1.types.DataStore.BillingEstimation):
	Output only. Data size estimation for
	billing.

[ffeldhaus]

@parthea Unfortunately this is only fixed in discoveryengine_v1 not in the other versions discoveryengine_v1beta and discoveryengine_v1alpha. Can you please fix it in all versions especially as some features are not yet available in v1.

[parthea]

I updated the internal bug to request that v1beta and v1alpha also be updated.