skip to main content
Using the Driver : Configuring and Connecting to Data Sources : Configuring the Product Using the GUI : Bulk Tab
  

Try DataDirect Drivers Now
Bulk Tab
The Bulk tab allows you to specify DataDirect Bulk Load data source settings. The fields are optional unless otherwise noted. On this tab, provide values for the options in the following table; then, click Apply.
See "Using DataDirect Bulk Load" for a general description of DataDirect Bulk Load.
Figure 7. Bulk tab
The Bulk tab of the ODBC Salesforce Driver Setup dialog box
Connection Options: Bulk
Description
Specifies whether the driver can use the Salesforce Bulk API for selects based on the value of the Bulk Fetch Threshold connection option. If the number of rows expected in the result set exceeds the value of Bulk Fetch Threshold option, the driver uses the Salesforce Bulk API to execute the select operation. Using the Salesforce Bulk API may significantly reduce the number of Web service calls used to execute a statement and, therefore, may improve performance.
If enabled, the driver can use the Salesforce Bulk API for selects based on the value of the Bulk Fetch Threshold connection option. If the number of rows expected in the result set exceeds the value of Bulk Fetch Threshold option, the driver uses the Salesforce Bulk API to execute the select operation.
If disabled, the driver does not use the Salesforce Bulk API, and the Bulk Fetch Threshold option is ignored.
Default: Enabled
If there is a TOP or LIMIT clause in the select query, the driver considers the TOP or LIMIT clause value as the expected number of rows in the result set and compares it with the value of the Bulk Fetch Threshold option to choose either Bulk API or REST API for executing the select query.
If the value of TOP or LIMIT clause is greater than that of the Bulk Fetch Threshold option, but the actual row count in the result set is less, the performance of the select query might be affected adversely.
Specifies whether the driver can use the Salesforce Bulk API for inserts, updates, and deletes based on the value of the Bulk Load Threshold connection option. If the number of affected rows exceeds the value of Bulk Load Threshold option, the driver uses the Salesforce Bulk API to execute the insert, update, or delete operation. Using the Salesforce Bulk API may significantly reduce the number of Web service calls used to execute a statement and, therefore, may improve performance.
If enabled, the driver can use the Salesforce Bulk API for inserts, updates, and deletes based on the value of the Bulk Load Threshold connection option. If the number of affected rows exceeds the value of Bulk Load Threshold option, the driver uses the Salesforce Bulk API to execute the insert, update, or delete operation.
If disabled, the driver does not use the Salesforce Bulk API, and the Bulk Load Threshold option is ignored.
Default: Enabled
Determines whether the driver treats bulk load operations as synchronous or asynchronous.
If disabled, bulk load operations are synchronous. The driver does not return from the function that invoked an operation until the operation is complete or the BulkLoadTimeout period has expired. If the operation times out, the driver returns an error.
If enabled, bulk load operations are asynchronous. The driver returns from the function that invoked an operation after the operation is submitted to the server. The driver does not verify the completion status of the bulk load operation.
Default: Disabled
Determines whether multiple batches associated with a bulk load operation are processed by Salesforce in parallel or one at a time.
If set to 0 - Serial, multiple batches associated with a bulk load operation are processed one at a time.
If set to 1 - Parallel, multiple batches associated with a bulk load operation are processed in parallel. The order in which the batches are processed can vary.
Default: 1 - Parallel
Determines when the driver uses bulk load for insert, update, delete, or batch operations.
If set to 0, the driver always uses bulk load to execute insert, update, delete, or batch operations.
If set to x, the driver only uses bulk load if the Enable Bulk Load option is enabled and the number of rows to be updated by an insert, update, delete, or batch operation exceeds the threshold. If the operation times out, the driver returns an error.
Default: 4000
The time, in seconds, that the driver waits for a Salesforce bulk job to complete.
A value of zero means there is no timeout.
Default: 0
The number of rows that the driver sends to the database at a time during bulk operations.
Default: 1024
Specifies the number of seconds the driver waits to request bulk operation status. This interval is used by the driver the first time it requests status and for all subsequent status requests.
Default: 10
Specifies the character that the driver will use to delimit the field entries in a bulk load data file.
Default: None
Specifies the character that the driver will use to delimit the record entries in a bulk load data file.
Default: None
Specifies a number of rows that, if exceeded, signals the driver to use the Salesforce Bulk API for select operations. For this behavior to take effect, the Enable Bulk Fetch option must be enabled.
Default: 30000 (rows)
Specifies whether the driver uses PK chunking for select operations. PK chunking breaks down bulk fetch operations into smaller, more manageable batches for improved performance.
If enabled, the driver uses PK chunking for select operations when the expected number of rows in the result set is greater than the values of the Bulk Fetch Threshold and Primary Key Chunk Size options. For this behavior to take effect, the Enable Bulk Fetch option must also be enabled.
If disabled, the driver does not use PK chunking when executing select operations, and the Primary Key Chunk Size option is ignored.
Default: Enabled
Specifies the size, in rows, of a primary key (PK) chunk when PK chunking has been enabled via the Enable Primary Key Chunking option. The Salesforce Bulk API splits the query into chunks of this size.
Default: 100000 (rows)
If your application is already coded to use parameter array batch functionality, you can leverage DataDirect Bulk Load features through the Enable Bulk Load connection option. Enabling this option automatically converts the parameter array batch operation to use the database bulk load protocol.
If you are not using parameter array batch functionality, you can export data to a bulk load data file, verify the metadata of the bulk load configuration file against the structure of the target table, and bulk load data to a table. Use the following steps to accomplish these tasks.
1. To export data from a table to a bulk load data file, click Export Table from the Bulk tab. The Export Table dialog box appears.
Figure 8. ODBC Salesforce Export Table Driver Setup dialog box
The Export Table Driver Setup of the ODBC Salesforce Driver Setup dialog box
Both a bulk data file and a bulk configuration file are produced by exporting a table. The configuration file has the same name as the data file, but with an XML extension. See "Using DataDirect Bulk Load" for details about these files.
The bulk export operation can create a log file and can also export to external files. See "External Overflow Files" for more information. The export operation can be configured such that if any errors or warnings occur:
*The operation always completes.
*The operation always terminates.
*The operation terminates after a certain threshold of warnings or errors is exceeded.
Table Name: A string that specifies the name of the source database table and, optionally, the columns containing the data to be exported. The driver uses the table name in the FROM clause of a SELECT * FROM tablename SQL statement. If you want to only export certain columns from your Salesforce table, then you can enter a SELECT statement in this field using the format:
(SELECT column1, column2, ... FROM tablename)
For example, to export data from the Salesforce ACCOUNT table excluding some of the audit columns, enter the following SQL in the Table Name field:
(SELECT SYS_NAME, TYPE, BILLINGSTREET, BILLINGCITY, BILLINGSTATE, BILLINGPOSTALCODE, BILLINGCOUNTRY,
SHIPPINGSTREET, SHIPPINGCITY, SHIPPINGSTATE, SHIPPINGPOSTALCODE, SHIPPINGCOUNTRY, PHONE, FAX, WEBSITE,
INDUSTRY, ANNUALREVENUE, NUMBEROFEMPLOYEES, DESCRIPTION FROM ACCOUNT)
Export Filename: A string that specifies the path (relative or absolute) and file of the bulk load data file to which the data is to be exported. It also specifies the file name of the bulk configuration file. The file name must be the fully qualified path to the bulk data file. These files must not already exist; if one of both of them already exists, an error is returned.
Log Filename: A string that specifies the path (relative or absolute) and file name of the bulk log file. The log file is created if it does not exist. The file name must be the fully qualified path to the log file. Events logged to this file are:
*Total number of rows fetched
*A message for each row that failed to export
*Total number of rows that failed to export
*Total number of rows successfully exported
Information about the load is written to this file, preceded by a header. Information about the next load is appended to the end of the file.
If you do not supply a value for Log Filename, no log file is created.
Error Tolerance: A value that specifies the number of errors to tolerate before an operation terminates. A value of 0 indicates that no errors are tolerated; the operation fails when the first error is encountered.
The default of -1 means that an infinite number of errors is tolerated.
Warning Tolerance: A value that specifies the number of warnings to tolerate before an operation terminates. A value of 0 indicates that no warnings are tolerated; the operation fails when the first warning is encountered.
The default of -1 means that an infinite number of warnings is tolerated.
Code Page: A value that specifies the code page value to which the driver must convert all data for storage in the bulk data file. See "Character Set Conversions" for more information.
The default value on Windows is the current code page of the machine.
Click Export Table to connect to the database and export data to the bulk data file or click Cancel.
2. To bulk load data from the bulk data file to a database table, click Load Table from the Bulk tab. The Load File dialog box appears.
Figure 9. ODBC Salesforce Load File Driver Setup dialog box
The Load File Driver Setup of the ODBC Salesforce Driver Setup dialog box
The load operation can create a log file and can also create a discard file that contains rows rejected during the load. The discard file is in the same format as the bulk load data file. After fixing reported issues in the discard file, the bulk load can be reissued using the discard file as the bulk load data file.
The export operation can be configured such that if any errors or warnings occur:
*The operation always completes.
*The operation always terminates.
*The operation terminates after a certain threshold of warnings or errors is exceeded.
If a load fails, the Load Start and Load Count options can be used to control which rows are loaded when a load is restarted after a failure.
Table Name: A string that specifies the name of the target database table and, optionally, the columns into which the data is loaded.
The fields defined in the load data file must have the same ordering of the fields defined in the Salesforce destination table. Because Salesforce defines additional audit columns that are managed by the database, your load data file may not contain data to load into these fields.
In this case, you can specify the exact columns that you want for the data to be inserted into using a Table Name string of the format:
table(column1, column2, ...)
For example, if your load data file contains only five fields of billing data that you wanted to load into the Salesforce ACCOUNT table, then the Table Name field would contain:
ACCOUNT(BILLINGSTREET, BILLINGCITY, BILLINGSTATE, BILLINGPOSTALCODE, BILLINGCOUNTRY)
Load Data Filename: A string that specifies the path (relative or absolute) and file name of the bulk data file from which the data is loaded.
Configuration Filename: A string that specifies the path (relative or absolute) and file name of the bulk configuration file.
Log Filename: A string that specifies the path (relative or absolute) and file name of the bulk log file. The file name must be the fully qualified path to the log file. Specifying a value for Log Filename creates the file if it does not already exist. Events logged to this file are:
*Total number of rows read
*Message for each row that failed to load
*Total number of rows that failed to load
*Total number of rows successfully loaded
Information about the load is written to this file, preceded by a header. Information about the next load is appended to the end of the file.
If you do not specify a value for Log Filename, no log file is created.
Discard Filename: A string that specifies the path (relative or absolute) and file name of the bulk discard file. Any row that cannot be inserted into the database as result of bulk load is added to this file, with the last row rejected added to the end of the file.
Information about the load is written to this file, preceded by a header. Information about the next load is appended to the end of the file.
If you do not specify a value for Discard Filename, a discard file is not created.
Error Tolerance: A value that specifies the number of errors to tolerate before an operation terminates. A value of 0 indicates that no errors are tolerated; the operation fails when the first error is encountered.
The default of -1 means that an infinite number of errors is tolerated.
Load Start: A value that specifies the first row to be loaded from the data file. Rows are numbered starting with 1. For example, when Load Start is 10, the first 9 rows of the file are skipped and the first row loaded is row 10. This option can be used to restart a load after a failure.
The default value is 1.
Read Buffer Size (KB): A value that specifies the size, in KB, of the buffer that is used to read the bulk data file for a bulk load operation.
The default value is 2048.
Warning Tolerance: A value that specifies the number of warnings to tolerate before an operation terminates. A value of 0 indicates that no warnings are tolerated; the operation fails when the first warning is encountered.
The default of -1 means that an infinite number of warnings is tolerated.
Load Count: A value that specifies the number of rows to be loaded from the data file. The bulk load operation loads rows up to the value of Load Count from the file to the database. It is valid for Load Count to specify more rows than exist in the data file. The bulk load operation completes successfully when either the number of rows specified by the Load Count value has been loaded or the end of the data file is reached. This option can be used in conjunction with Load Start to restart a load after a failure.
The default value is the maximum value for SQLULEN. If set to 0, no rows are loaded.
Click Load Table to connect to the database and load the table or click Cancel.
If you finished configuring your driver, proceed to Step 6 in Configuring the Product Using the GUI. Optionally, you can further configure your driver by clicking on the following tabs. The following sections provide details on the fields specific to each configuration tab:
*General tab allows you to configure options that are required for creating a data source.
*SQL Engine tab allows you to configure the SQL Engine's behavior.
*Advanced tab allows you to configure advanced behavior.
*Web Service tab allows you to configure the behavior of communications between the driver and web service.
*Pooling tab allows you to configure connection pooling behavior.