skip to main content
JDBC extensions : DDBulkLoad interface
  

Try DataDirect Drivers Now

DDBulkLoad interface

The DDBulkLoad interface allows your application to perform bulk load operations using DataDirect Bulk Load. The following 6.0 drivers support DataDirect Bulk Load:
*Oracle
*Salesforce
*SQL Server
DDBulkLoad Methods
Description
void clearWarnings()
Clears all warnings that were generated by this DDBulkLoad object.
void close()
Releases a DDBulkLoad object’s resources immediately instead of waiting for the connection to close.
long export(File)
Exports all rows from the table into the specified CSV file specified by a file reference. The table is specified using the setTableName() method. If the CSV file does not already exist, the driver creates it when the export() method is executed. In addition, the driver creates a bulk load configuration file matching the CSV file. For more information, refer to "Exporting Data to a CSV File" in the user’s guide for your driver.
This method also returns the number of rows that were successfully exported from the table.
long export(ResultSet, File)
Exports all rows from the specified ResultSet into the CSV file specified by a file reference. If the CSV file does not already exist, the driver creates it when the export() method is executed. In addition, the driver creates a bulk load configuration file matching the CSV file. For more information, refer to "Exporting Data to a CSV File" in the user’s guide for your driver.
This method also returns the number of rows that were successfully exported from the ResultSet object.
long export(String)
Exports all rows from the table into the CSV file specified by name. The table is specified using the setTableName() method. If the CSV file does not already exist, the driver creates it when the export() method is executed. In addition, the driver creates a bulk load configuration file matching the CSV file. For more information, refer to "Exporting Data to a CSV File" in the user’s guide for your driver.
This method also returns the number of rows that were successfully exported from the table.
long getBatchSize()
Returns the number of rows that the driver sends at a time when bulk loading data.
long getBinaryThreshold()
Returns the maximum size (in bytes) of binary data that can be exported to the CSV file. Once this size is reached, binary data is written to one or multiple external overflow files. For more information, refer to "External Overflow Files" in the user’s guide .
long getCharacterThreshold()
Returns the maximum size (in bytes) of character data that can be exported to the CSV file. Once this size is reached, character data is written to one or multiple external overflow files. for more information, refer to "External Overflow Files" in the user’s guide for your driver.
String getCodePage()
Returns the code page that the driver uses for the CSV file. For more information, refer to "Character Set Conversions" in the user’s guide for your driver.
String getConfigFile()
Returns the name of the bulk load configuration file. For more information, refer to "Bulk Load Configuration File" in the User’s Guide for your driver.
String getDiscardFile()
Returns the name of the discard file. The discard file contains rows that were unable to be loaded as the result of a bulk load operation. For more information, refer to "Discard File" in the user’s guide for your driver.
long getErrorTolerance()
Returns the number of errors that can occur before this DDBulkLoad object ends the bulk load operation.
String getLogFile()
Returns the name of the log file. The log file records information about each bulk load operation. For more information, refer to "Logging" in the user’s guide for your driver.
long getNumRows()
Returns the maximum number of rows from the CSV file or ResultSet object the driver will load when the load() method is executed.
Properties getProperties()
Returns the properties specified for a DDBulkLoad object. Properties are specified using the setProperties() method.
long getReadBufferSize()
Returns the size (in KB) of the buffer that is used to read the CSV file.
long getStartPosition()
Returns the position (number of the row) in a CSV file or ResultSet object from which the driver starts loading. The position is specified using the setStartPosition() method.
void getTableName()
Returns the name of the table to which the data is loaded into or exported from. For more information.Refer to "Loading Data From a ResultSet Object," "Loading Data From a CSV File," and "Exporting Data to a CSV File" in the user's guide for your driver.
long getTimeout()
Returns the number of seconds the bulk load operation requires to complete before it times out. The timeout is specified using the setTimeout() method.
SQLWarning getWarnings()
Returns any warnings generated by this DDBulkLoad object.
long getWarningTolerance()
Returns the maximum number of warnings that can occur. Once the maximum number is reached, the bulk load operation ends.
long load(File)
Loads data from the CSV file specified by a file reference into a table. The table is specified using the setTableName() method. This method also returns the number of rows that have been successfully loaded.
If logging is enabled using the setLogFile() method, information about the bulk load operation is recorded in the log file. If a discard file is created using the setDiscardFile() method, rows that were unable to be loaded are recorded in the discard file. For more information, refer to "Logging" and "Discard File" in the User’s Guide for your driver.
Before the bulk load operation is performed, your application can verify that the data in the CSV file is compatible with the structure of the target table using the validateTableFromFile() method.
long load(String)
Loads data from the CSV file specified by file name into a table. The table is specified using the setTableName() method. This method also returns the number of rows that have been successfully loaded.
If logging is enabled using the setLogFile() method, information about the bulk load operation is recorded in the log file. If a discard file is created using the setDiscardFile() method, rows that were unable to be loaded are recorded in the discard file. For more information, refer to "Logging" and "Discard File" in the user’s guide for your driver.
Before the bulk load operation is performed, your application can verify that the data in the CSV file is compatible with the structure of the target table using the validateTableFromFile() method.
long load(ResultSet)
Loads data from a ResultSet object into the table specified using the setTableName() method. This method also returns the number of rows that have been successfully loaded.
If logging is enabled using the setLogFile() method, information about the bulk load operation is recorded in the log file. For more information, refer to "Logging" in the user’s guide for your driver.
The structure of the table that produced the ResultSet object must match the structure of the target table. If not, the driver throws an exception.
void setBatchSize(long)
Specifies the number of rows that the driver sends at a time when bulk loading data. Performance can be improved by increasing the number of rows the driver loads at a time because fewer network round trips are required. Be aware that increasing the number of rows that are loaded also causes the driver to consume more memory on the client.
If unspecified, the driver uses a value of 2048.
void setBinaryThreshold(long)
Specifies the maximum size (in bytes) of binary data to be exported to the CSV file. Any column with data over this threshold is exported into individual external overflow files and a marker of the format {DD LOBFILE "filename"} is placed in the CSV file to signify that the data for this column is located in an external file. The format for overflow file names is:
csv_filename_xxxxxx.lob
where:
csv_filename
is the name of the CSV file.
xxxxxx
is a 6-digit number that increments the overflow file.
For example, if multiple overflow files are created for a CSV file named CSV1, the file names would look like this:
CSV1.000001.lob
CSV1.000002.lob
CSV1.000003.lob
...
If set to -1, the driver does not overflow binary data to external files. If unspecified, the driver uses a value of 4096.
For more information, refer to "External Overflow Files" in the user’s guide for your driver.
void setCharacterThreshold(long)
Specifies the maximum size (in bytes) of character data to be exported to the CSV file. Any column with data over this threshold is exported into individual external overflow files and a marker of the format {DD LOBFILE "filename"} is placed in the CSV file to signify that the data for this column is located in an external file. The format for overflow file names is:
csv_filename_xxxxxx.lob
where:
csv_filename
is the name of the CSV file.
xxxxxx
is a 6-digit number that increments the overflow file.
For example, if multiple overflow files are created for a CSV file named CSV1, the file names would look like this:
CSV1.000001.lob
CSV1.000002.lob
CSV1.000003.lob
...
If set to -1, the driver does not overflow character data to external files.If unspecified, the driver uses a value of 4096.
For more information, refer to "External Overflow Files" in the user’s guide for your driver.
void setCodePage(String)
Specifies the code page the driver uses for the CSV file. For more information, refer to "Character Set Conversions" in the user’s guide for your driver.
void setConfigFile(String)
Specifies the fully qualified directory and file name of the bulk load configuration file. If the Column Info section in the bulk load configuration file is specified, the driver uses it to map the columns in the CSV file to the columns in the target table when performing a bulk load operation.
If unspecified, the name of the bulk load configuration file is assumed to be csv_filename.xml, where csv_filename is the file name of the CSV file.
If set to an empty string, the driver does not try to use the bulk load configuration file and reads all data from the CSV file as character data.
For more information, refer to "Bulk Load Configuration File" the user’s guide for your driver.
void setDiscardFile(String)
Specifies the fully qualified directory and file name of the discard file. The discard file contains rows that were unable to be loaded from a CSV file as the result of a bulk load operation. After fixing the reported issues in the discard file, the bulk load can be reissued, using the discard file as the CSV file. If unspecified, a discard file is not created. For more information, refer to "Discard File" in the user’s guide for your driver.
void setErrorTolerance(long)
Specifies the maximum number of errors that can occur. Once the maximum number is reached, the bulk load operation ends. Errors are written to the log file. If set to 0, no errors are tolerated; the bulk load operation fails if any error is encountered. Any rows that were processed before the error occurred are loaded. If unspecified or set to -1, an infinite number of errors are tolerated.
void setLogFile(String)
Specifies the fully qualified directory and file name of the log file. The log file records information about each bulk load operation.If unspecified, a log file is not created. For more information, refer to "Logging" in the user’s guide for your driver.
void setNumRows()
Specifies the maximum number of rows from the CSV file or ResultSet object the driver will load.
void setProperties(Properties)
Specifies one or more of the following properties for a DDBulkLoad object:
tableName numRows
codePage binaryThreshold
timeout characterThreshold
logFile errorTolerance
discardFile warningTolerance
configFile readBufferSize
startPosition batchSize
operation
Except for the operation property, these properties also can be set using the corresponding setxxx() methods, which provide a description of the values that can be set.
The operation property defines which type of bulk operation will be performed when a load method is called. The operation property accepts the following values: insert, update, delete, or upsert. The default value is insert. For more information, refer to "Specifying the Bulk Load Operation" in the user’s guide for your driver.
void setReadBufferSize(long)
Specifies the size (in KB) of the buffer that is used to read the CSV file. If unspecified, the driver uses a value of 2048.
void setStartPosition()
Specifies the position (number of the row) in a CSV file or ResultSet object from which the bulk load operation starts. For example, if a value of 10 is specified, the first 9 rows of the CSV file are skipped and the first row loaded is row 10.
void setTableName(tablename ([destinationColumnList]))
When loading data into a table, specifies the name of the table into which the data is loaded (tablename).
Optionally, for the Salesforce driver, you can specify the column names that identify which columns to update in the table (destinationColumnList). Specifying column names is useful when loading data from a CSV file into a table. The column names used in the column list must be the names reported by the driver for the columns in the table. For example, if you are loading data into the Salesforce system column NAME, the column list must identify the column as SYS_NAME.
If destinationColumnList is not specified, a one-to-one mapping is performed between the columns in the CSV file and the columns in the table.
destinationColumnList has the following format:
(destColumnName [,destColumnName]...)
where:
destColumnName
is the name of the column in the table to be updated.
The number of specified columns must match the number of columns in the CSV file. For example, the following call tells the driver to update the Name, Address, City, State, PostalCode, Phone, and Website columns:
bulkload.setTableName("account(Name, Address, City,State, PostalCode, Phone, Website)")
When exporting data from a table, specifies the name of the table from which the data is exported. If the specified table does not exist, the driver throws an exception. For more information, refer to "Loading Data From a ResultSet Object," "Loading Data From a CSV File," and "Exporting Data to a CSV File" in the user’s guide for your driver.
void setTimeout(long)
Sets the maximum number of seconds that can elapse for this bulk load operation to complete. Once this number is reached, the bulk load operation times out.
void setWarningTolerance(long)
Specifies the maximum number of warnings that can occur. Once the maximum is reached, the bulk load operation ends. Warnings are written to the log file.
If set to 0, no warnings are tolerated; the bulk load operation fails if any warning is encountered.
If unspecified or set to -1, an infinite number of warnings are tolerated.
Properties validateTableFromFile()
Verifies the metadata in the bulk load configuration file against the structure of the table to which the data is loaded. This method is used to ensure that the data in a CSV file is compatible with the structure of the target table before the actual bulk load operation is performed. The driver performs checks to detect mismatches of the following types:
Data types
Column sizes
Code pages
Column info
This method returns a Properties object with an entry for each of these checks:
*If no mismatches are found, the Properties object does not contain any messages.
*If minor mismatches are found, the Properties object lists the problems.
*If problems are detected that would prevent a successful bulk load operation, for example, if the target table does not exist, the driver throws an exception.
For more information, refer to "Verifying the Bulk Load Configuration File for Database Connections" in the user’s guide for your driver.