Configuring throttling for OData queries

Hybrid Data Pipeline supports throttling OData queries. By default, when executing an OData query, Hybrid Data Pipeline sends the query to the backend data store. All of the results are then fetched and persisted, and the first page of results is returned to the application. Having multiple large queries running simultaneously can consume system resources, which can negatively impact the performance of the service for all users. Further, some results are never fully viewed by applications, meaning that resources are unnecessarily allocated to return unused data. As a result, an administrator may want to limit the number of active OData queries to throttle the amount of resources consumed.

OData query throttling may be configured with the ODataMaxConcurrentQueries limit. When the ODataMaxConcurrentQueries limit is set to 0 (zero), there is no maximum number of persisted queries. When ODataMaxConcurrentQueries is set to a positive integer, rows are fetched one page in advance of application requests. This maintains quick response times in addition to minimizing the expense associated with executing large queries. Queries that contain more than one page of results are persisted in system memory until completely returned to the application or terminated. To prevent users from exhausting system and database resources, the maximum number of persisted queries are limited to the value specified. When this limit is exceeded, the least recently used query is canceled, and subsequent attempts to retrieve data from the canceled query will fail.

The ODataMaxConcurrentQueries limit can be applied at four levels: system, tenant, user, and data source. The following hierarchy applies to these levels.

A query limit set on a data source overrides any query limit set at the other levels; a query limit set on a user account overrides a query limit set on a tenant or at the system level; a query limit set on a tenant overrides a query limit set at the system level; and a query limit set at the system level overrides the default query limit. Default and system limits apply to behavior across Hybrid Data Pipeline, while limits on data sources, users, and tenants apply to the resources they handle.

To configure OData query throttling, the administrator must have either the Administrator (12) or the Limits (27) permission.

OData query throttling can be configured at the system level either with the Web UI or with the Limits API. For details on using the Web UI, see Manage Limits view.

The following POST creates a system-level limit of 50000 queries. The number 6 is the ID of the ODataMaxConcurrentQueries limit. The payload passes 50000 as the value for this limit.

OData query throttling can be configured at the tenant level either with the Web UI or with the Limits API. For details on using the Web UI, see Manage Limits view.

The following POST sets a limit of 60000 queries on the specified tenant. The number 32 is the ID of the tenant, and the number 6 is the ID of the ODataMaxConcurrentQueries limit. The payload passes 60000 as the value for this limit.

OData query throttling can be configured at the user level either with the Web UI or with the Limits API. For details on using the Web UI, see Manage Users view.

The following POST sets a limit of 3000 queries on the specified user account. The number 86 is the ID of the user account, and the number 6 is the ID of the ODataMaxConcurrentQueries limit. The payload passes 3000 as the value for this limit.

OData query throttling can only be configured at the data sources level using the Limits API. The following POST sets a limit of 10000 queries on the specified data source. The number 86 is the ID of the user account; the number 14 is the ID of the data source that is owned by the user account; and the number 6 is the ID of the ODataMaxConcurrentQueries limit. The payload passes 10000 as the value for this limit.