Hybrid Data Pipeline divides results that exceed a threshold into multiple pages. For OData queries, you can use server-side or client-side pagination:
By default, Hybrid Data Pipeline divides OData responses with a maximum of 2000 top-level entities per response. If the response is larger than 2000 entities, the first page contains the first 2000 entities and contains a next link at the end of the response. The next link contains the URL to fetch the next page of results. Next link URLs should be passed back without modification. You can modify the maximum number of entities returned in a page by setting the OData Page Size data source parameter as described in Configuring data sources for OData Version 4 connectivity.
Client-side pagination is controlled by both the client and the Hybrid Data Pipeline OData service. Requests can specify a particular page size with the $top query parameter and can navigate through the pages by specifying different values for the $skip query parameter. The Top Mode setting allows the Hybrid Data Pipeline service to optimize queries in certain situations. You can set the Top Mode in the data source definition or use the ddcloud-top-mode header in requests to inform the service of how the client uses $top. See Configuring data sources for OData Version 4 connectivity and Top Mode for more information.
For example, the following URL requests Employees entities in pages of 100.
The client can request any page size it needs. However, the Hybrid Data Pipeline connectivity service might return fewer entities than were requested. In this case, the response will contain a next link, as with server-side paging. The client should use the next link(s) to get all of the results before requesting the next page.
By default, Hybrid Data Pipeline divides OData responses with a maximum of 2000 top-level entities per response. If the response is larger than 2000 entities, the first page contains the first 2000 entities and contains a next link at the end of the response. The next link contains the URL to fetch the next page of results. Next link URLs should be passed back without modification. You can modify the maximum number of entities returned in a page by setting the OData Page Size data source parameter as described in Configuring data sources for OData Version 4 connectivity.