Diagnostic Tools for Java Developers

Article: dts0147

Applies to: dtSearch Engine Web

The dtSearch Text Retrieval Engine includes several diagnostic tools to help isolate problems that arise during execution of a program. These include: (1) detailed error reporting through the dtsErrorInfo structure, (2) debug logging to a text file, and (3) generation of a stack trace when a program crashes, and (4) log files created in the index folder.

Error Reporting

Most of the dtSearch Engine's API functions involve a "Job" structure of some type (a SearchJob, an IndexJob, etc.). After a Job has been executed, the Errors property of the job will contain a JobErrorInfo that provides the text and an error code for each error that occurred.

Debug Logging

dtSearch Engine can generate a debug log that provides detailed information about internal processing of an API call. To enable debug logging in the dtSearch Engine, call the SetDebugLogging method of the Server object.

setDebugLogging(LogName as String, flags as Long)

SetDebugLogging tells the dtSearch Engine to start recording a debug log in the specified file. The flags argument controls optional logging behavior. To turn off debug logging, call SetDebugLogging with a blank LogName.

Flag	Effect
dtsLogTime	Each line in the log will have include the time, in hundredths of a second, since the start of execution.
dtsLogCommit	The log will be committed to disk after each line is written. This slows execution considerably but ensures that the log will survive a system crash.
dtsLogAppend	Log data is appended to the file if it already exists. Otherwise the file will be overwritten.

Note:  Debug logging will not work if the process creating the log lacks write access to the folder containing the log file.   For example, a web application running as the logged-on user may not be permitted to write to some folders.  If debug logging does not seem to work,

(1) Create an empty folder with a simple name like c:\log

(2) Right-click the c:\log folder in Windows Explorer and select "Permissions"

(3) Set the permissions for c:\log to grant "Everyone" "Full Control" access

(4) Use c:\log\debuglog.txt as the name of the log file

Crash Diagnostics

The dtSearch Engine can set up a crash handler to generate a stack trace if an unhandled exception occurs during execution of the program.

To set up a crash handler, call SetDebugLogging (as above) with the dtsCrashLog flag. After a crash occurs, the text file will contain a stack trace file identifying the state of the calling program and the dtSearch Engine when the crash occurred. 

Index Log Files

After an index update completes, the file indexlog.dat in the index folder will contain a text record of what happened during the index update.   The log includes IndexJob settings, the dtSearch Engine build number, the start and end times, the option settings in effect, and a summary of the number of files indexed, size of the index, etc.  The indexlog.dat file will also contain a list of files that could not be indexed, along with a diagnostic message for each file.

The file history.ix contains a chronological record of every index update, similar to the indexlog.dat file but without the list of files that could not be indexed.  The history.ix file is limited in size so it will include only the most recent updates for indexes that have been updated many times.

Both log files are in text format (the encoding is UTF-8).

dtSearch Desktop creates additional HTML log files, Index_LastUpdateErrors.htm and Index_LastUpdateSummary.htm.  The information needed to generate this log is obtained from the index status monitoring APIs, which are documented here:
https://support.dtsearch.com/webhelp/dtsearchCppApi/Monitoring_Index_Jobs.html