Online Store Spec

class databricks.ml_features.online_store_spec.AmazonDynamoDBSpec(*, region: Optional[str], access_key_id: Optional[str] = None, secret_access_key: Optional[str] = None, session_token: Optional[str] = None, table_name: Optional[str] = None, read_secret_prefix: Optional[str] = None, write_secret_prefix: Optional[str] = None, ttl: Optional[datetime.timedelta] = None, endpoint_url: Optional[str] = None)

Bases: databricks.ml_features.online_store_spec.online_store_spec.OnlineStoreSpec

Note

Aliases: databricks.feature_engineering.online_store_spec.AmazonDynamoDBSpec, databricks.feature_store.online_store_spec.AmazonDynamoDBSpec

This OnlineStoreSpec implementation is intended for publishing features to Amazon DynamoDB.

If table_name is not provided, publish_table() will use the offline store’s database and table name combined as the online table name.

To use a different table name in the online store, provide a value for the table_name argument.

The expected read or write secrets for DynamoDB for a given {prefix} string are ${prefix}-access-key-id, ${prefix}-secret-access-key, and ${prefix}-session-token.

If none of the access_key_id, secret_access_key, and write_secret_prefix are passed, the instance profile attached to the cluster will be used to write to DynamoDB.

Parameters:
  • region – Region to access online store.
  • access_key_id – Access key ID that has access to the online store. Deprecated. Use write_secret_prefix instead.
  • secret_access_key – Secret access key to access the online store. Deprecated. Use write_secret_prefix instead.
  • session_token – Session token to access the online store. Deprecated. Use write_secret_prefix instead.
  • table_name – Table name.
  • read_secret_prefix – Prefix for read secret.
  • write_secret_prefix – Prefix for write secret.
  • ttl – The time to live for data published to the online store. This attribute is only applicable when publishing time series feature tables. If the time to live is specified for a time series table, publish_table() will publish a window of data instead of the latest snapshot.
access_key_id

Warning

databricks.ml_features.online_store_spec.amazon_dynamodb_online_store_spec.AmazonDynamoDBSpec.access_key_id is deprecated. This method will be removed in a future release. Use write_secret_prefix instead.

Access key ID that has access to the online store. Property will be empty if write_secret_prefix or the instance profile attached to the cluster are intended to be used.

secret_access_key

Warning

databricks.ml_features.online_store_spec.amazon_dynamodb_online_store_spec.AmazonDynamoDBSpec.secret_access_key is deprecated. This method will be removed in a future release. Use write_secret_prefix instead.

Secret access key to access the online store. Property will be empty if write_secret_prefix or the instance profile attached to the cluster are intended to be used.

session_token

Warning

databricks.ml_features.online_store_spec.amazon_dynamodb_online_store_spec.AmazonDynamoDBSpec.session_token is deprecated. This method will be removed in a future release. Use write_secret_prefix instead.

Session token to access the online store. Property will be empty if write_secret_prefix or the instance profile attached to the cluster are intended to be used.

endpoint_url

Endpoint url of DynamoDB online store, mainly used for testing with LocalStack

cloud

Define the cloud property for the data store.

store_type

Define the data store type.

region

Region to access the online store.

ttl

Time to live attribute for the online store.

auth_type()

Publish Auth type.

class databricks.ml_features.online_store_spec.AmazonRdsMySqlSpec(hostname: str, port: int, user: Optional[str] = None, password: Optional[str] = None, database_name: Optional[str] = None, table_name: Optional[str] = None, driver_name: Optional[str] = None, read_secret_prefix: Optional[str] = None, write_secret_prefix: Optional[str] = None)

Bases: databricks.ml_features.online_store_spec.online_store_spec.OnlineStoreSpec

Note

Aliases: databricks.feature_engineering.online_store_spec.AmazonRdsMySqlSpec, databricks.feature_store.online_store_spec.AmazonRdsMySqlSpec

Class that defines and creates AmazonRdsMySqlSpec objects.

This OnlineStoreSpec implementation is intended for publishing features to Amazon RDS MySQL and Aurora (MySQL-compatible edition).

See OnlineStoreSpec documentation for more usage information, including parameter descriptions.

Parameters:
  • hostname – Hostname to access online store.
  • port – Port number to access online store.
  • user – Username that has access to the online store. Deprecated. Use write_secret_prefix instead.
  • password – Password to access the online store. Deprecated. Use write_secret_prefix instead.
  • database_name – Database name.
  • table_name – Table name.
  • driver_name – Name of custom JDBC driver to access the online store.
  • read_secret_prefix – Prefix for read secret.
  • write_secret_prefix – Prefix for write secret.
hostname

Hostname to access the online store.

port

Port number to access the online store.

database_name

Database name.

cloud

Define the cloud propert for the data store.

store_type

Define the data store type property.

auth_type()

Publish Auth type.

class databricks.ml_features.online_store_spec.AzureCosmosDBSpec(*, account_uri: str, database_name: Optional[str] = None, container_name: Optional[str] = None, read_secret_prefix: Optional[str] = None, write_secret_prefix: str, **kwargs)

Bases: databricks.ml_features.online_store_spec.online_store_spec.OnlineStoreSpec

Note

Aliases: databricks.feature_engineering.online_store_spec.AzureCosmosDBSpec, databricks.feature_store.online_store_spec.AzureCosmosDBSpec

This OnlineStoreSpec implementation is intended for publishing features to Azure Cosmos DB.

If database_name and container_name are not provided, publish_table() will use the offline store’s database and table name as the Cosmos DB database and container name.

The expected read or write secret for Cosmos DB for a given {prefix} string is ${prefix}-authorization-key.

The authorization key can be either the Cosmos DB account primary or secondary key.

Parameters:
  • account_uri – URI of the Cosmos DB account.
  • database_name – Database name.
  • container_name – Container name.
  • read_secret_prefix – Prefix for read secret.
  • write_secret_prefix – Prefix for write secret.
account_uri

Account URI of the online store.

database_name

Database name.

container_name

Container name.

target_throughput_threshold_for_provisioned

Threshold for handling CosmosDB Requests Units. Note that this is for CosmosDB Provisioned Throughput, so you need to specify a number between 0 and 1 indicating the percentage.

target_throughput_for_serverless

Threshold for handling CosmosDB Requests Units. Note that this is for CosmosDB Serverless account, so you need to specify an absolute number, which is the threshold for the Spark job to write to the account.

cloud

Define the cloud property for the data store.

store_type

Define the data store type.

auth_type()

Publish Auth type.

class databricks.ml_features.online_store_spec.AzureMySqlSpec(hostname: str, port: int, user: Optional[str] = None, password: Optional[str] = None, database_name: Optional[str] = None, table_name: Optional[str] = None, driver_name: Optional[str] = None, read_secret_prefix: Optional[str] = None, write_secret_prefix: Optional[str] = None)

Bases: databricks.ml_features.online_store_spec.online_store_spec.OnlineStoreSpec

Note

Aliases: databricks.feature_engineering.online_store_spec.AzureMySqlSpec, databricks.feature_store.online_store_spec.AzureMySqlSpec

Define the AzureMySqlSpec class.

This OnlineStoreSpec implementation is intended for publishing features to Azure Database for MySQL.

See OnlineStoreSpec documentation for more usage information, including parameter descriptions.

Parameters:
  • hostname – Hostname to access online store.
  • port – Port number to access online store.
  • user – Username that has access to the online store. Deprecated. Use write_secret_prefix instead.
  • password – Password to access the online store. Deprecated. Use write_secret_prefix instead.
  • database_name – Database name.
  • table_name – Table name.
  • driver_name – Name of custom JDBC driver to access the online store.
  • read_secret_prefix – Prefix for read secret.
  • write_secret_prefix – Prefix for write secret.
hostname

Hostname to access the online store.

port

Port number to access the online store.

database_name

Database name.

cloud

Define the cloud the fature store runs.

store_type

Define the data store type.

auth_type()

Publish Auth type.

class databricks.ml_features.online_store_spec.AzureSqlServerSpec(hostname: str, port: int, user: Optional[str] = None, password: Optional[str] = None, database_name: Optional[str] = None, table_name: Optional[str] = None, driver_name: Optional[str] = None, read_secret_prefix: Optional[str] = None, write_secret_prefix: Optional[str] = None)

Bases: databricks.ml_features.online_store_spec.online_store_spec.OnlineStoreSpec

Note

Aliases: databricks.feature_engineering.online_store_spec.AzureSqlServerSpec, databricks.feature_store.online_store_spec.AzureSqlServerSpec

This OnlineStoreSpec implementation is intended for publishing features to Azure SQL Database (SQL Server).

The spec supports SQL Server 2019 and newer.

See OnlineStoreSpec documentation for more usage information, including parameter descriptions.

Parameters:
  • hostname – Hostname to access online store.
  • port – Port number to access online store.
  • user – Username that has access to the online store. Deprecated. Use write_secret_prefix instead.
  • password – Password to access the online store. Deprecated. Use write_secret_prefix instead.
  • database_name – Database name.
  • table_name – Table name.
  • driver_name – Name of custom JDBC driver to access the online store.
  • read_secret_prefix – Prefix for read secret.
  • write_secret_prefix – Prefix for write secret.
hostname

Hostname to access the online store.

port

Port number to access the online store.

database_name

Database name.

cloud

Define the cloud the fature store runs.

store_type

Define the data store type.

auth_type()

Publish Auth type.

class databricks.ml_features.online_store_spec.OnlineStoreSpec(_type, hostname: [<class 'str'>, None] = None, port: [<class 'int'>, None] = None, user: Optional[str] = None, password: Optional[str] = None, database_name: Optional[str] = None, table_name: Optional[str] = None, driver_name: Optional[str] = None, read_secret_prefix: Optional[str] = None, write_secret_prefix: Optional[str] = None, _internal_properties: Optional[Dict[str, str]] = None)

Bases: abc.ABC

Note

Aliases: databricks.feature_engineering.online_store_spec.OnlineStoreSpec, databricks.feature_store.online_store_spec.OnlineStoreSpec

Parent class for all types of OnlineStoreSpec objects.

Abstract base class for classes that specify the online store to publish to.

If database_name and table_name are not provided, publish_table() will use the offline store’s database and table names.

To use a different database and table name in the online store, provide values for both database_name and table_name arguments.

The JDBC driver can be customized with the optional driver_name argument. Otherwise, a default is used.

Strings in the primary key should not exceed 100 characters.

The online database should already exist.

Note

It is strongly suggested (but not required), to provide read-only database credentials via the read_secret_prefix in order to grant the least amount of database access privileges to the served model. When providing a read_secret_prefix, the secrets must exist in the scope name using the expected format, otherwise publish_table() will return an error.

Parameters:
  • hostname – Hostname to access online store. The database hostname cannot be changed. Subsequent publish calls to the same online store must provide the same hostname.
  • port – Port number to access online store. The database port cannot be changed. Subsequent publish calls to the same online store must provide the same port.
  • user – Username that has write access to the online store. Deprecated. Use write_secret_prefix instead.
  • password – Password to access the online store. Deprecated. Use write_secret_prefix instead.
  • database_name – Database name.
  • table_name – Table name.
  • driver_name – Name of custom JDBC driver to access the online store.
  • read_secret_prefix –

    The secret scope name and secret key name prefix where read-only online store credentials are stored. These credentials will be used during online feature serving to connect to the online store from the served model. The format of this parameter should be ${scope-name}/${prefix}, which is the name of the secret scope, followed by a /, followed by the secret key name prefix. The scope passed in must contain the following keys and corresponding values:

    • ${prefix}-user where ${prefix} is the value passed into this function. For example if this function is called with datascience/staging, the datascience secret scope should contain the secret named staging-user, which points to a secret value with the database username for the online store.
    • ${prefix}-password where ${prefix} is the value passed into this function. For example if this function is called with datascience/staging, the datascience secret scope should contain the secret named staging-password, which points to a secret value with the database password for the online store.

    Once the read_secret_prefix is set for an online store, it cannot be changed.

  • write_secret_prefix –

    The secret scope name and secret key name prefix where read-write online store credentials are stored. These credentials will be used to connect to the online store to publish features. If user and password are passed, this field must be None, or an exception will be raised. The format of this parameter should be ${scope-name}/${prefix}, which is the name of the secret scope, followed by a /, followed by the secret key name prefix. The scope passed in must contain the following keys and corresponding values:

    • ${prefix}-user where ${prefix} is the value passed into this function. For example if this function is called with datascience/staging, the datascience secret scope should contain the secret named staging-user, which points to a secret value with the database username for the online store.
    • ${prefix}-password where ${prefix} is the value passed into this function. For example if this function is called with datascience/staging, the datascience secret scope should contain the secret named staging-password, which points to a secret value with the database password for the online store.
type

Type of the online store.

table_name

Table name.

user

Warning

databricks.ml_features.online_store_spec.online_store_spec.OnlineStoreSpec.user is deprecated. This method will be removed in a future release. Use write_secret_prefix instead.

Username that has access to the online store.

Property will be empty if write_secret_prefix argument was used.
password

Warning

databricks.ml_features.online_store_spec.online_store_spec.OnlineStoreSpec.password is deprecated. This method will be removed in a future release. Use write_secret_prefix instead.

Password to access the online store.

Property will be empty if write_secret_prefix argument was used.
driver

Name of the custom JDBC driver to access the online store.

read_secret_prefix

Prefix for read access to online store.

Name of the secret scope and prefix that contains the username and password to access the online store with read-only credentials.

See the read_secret_prefix parameter description for details.

write_secret_prefix

Secret prefix that contains online store login info.

Name of the secret scope and prefix that contains the username and password to access the online store with read/write credentials. See the write_secret_prefix parameter description for details.

cloud

Cloud provider where this online store is located.

store_type

Store type.

auth_type()

Publish Auth type.