Large dataset handling

Overview

The API provides mechanisms to handle large datasets efficiently, both for retrieving and sending data. These mechanisms help prevent timeouts and ensure optimal performance.

Pagination for GET Requests

The API supports two types of pagination:

1. Offset-Based Pagination

Most endpoints use offset-based pagination with the following parameters:

Note: Each endpoint defines its own default values for these parameters if not specified.

Response format includes a pageInfo object:

{
  "pageInfo": {
    "pageSize": "number",    // Records per page
    "currentPage": "number", // Current page number
    "rowCount": "number"     // Total number of records (null if not available)
  }
}

2. Cursor-Based Pagination

Some endpoints, use cursor-based pagination for improved performance with large datasets. This approach:

• Provides more consistent performance
• Handles concurrent updates better
• Is ideal for real-time data feeds

Note: When using cursor-based endpoints, refer to the specific endpoint documentation for pagination parameters and response format.

Size Limits for POST Requests

Bulk Operations

When sending large datasets for bulk operations:

• Maximum request size: 20MB
• Exceeding this limit returns 413 - Payload Too Large