Using dtSearch products with cloud storage (OneDrive, Amazon S3, etc.)

Last Reviewed: July 11, 2020

Article: DTS0243

Applies to: dtSearch Desktop 7, dtSearch Engine 7

dtSearch Desktop/Network

dtSearch Desktop/Network can index local and network folders, and most cloud storage services provide a way to access documents through a network share or locally mounted folder.  
For example, DropBox will add a "DropBox" folder on your system and OneDrive will add a "OneDrive" folder on your system.  To index these types of folders, click "Add Folder" in the Update Index dialog box.  Typically this will add a folder named C:\Users\<your name>\DropBox or C:\Users\<your name>\OneDrive to the index, which is where this data is stored locally.

Some cloud storage services instead or alternatively provide access through a network share.  For example, you can set up OneDrive for Business to be accessible through a network share.  To index this type of data, add the network share to the index, again using "Add Folder".  The network share will have a UNC format, like this: \\server\share\folder.

dtSearch Engine

(1) File system indexing

The options described above for indexing using dtSearch Desktop to index a mounted folder or network share will also work with the dtSearch Engine.   

(2) Direct integration with the cloud storage API.

The dtSearch Engine has a "DataSource" API that is designed to enable developers to connect to content that is accessible through an API.  Most commonly this is used with databases but the API is completely general and can be used with any content.  Each logical document indexed can consist any combination of plain or Unicode text, a set of field-value pairs, and a binary document file to be indexed using the dtSearch Engine's built-in document filters.  The file can be provided as a disk file, memory buffer, or stream object.

If a cloud storage service has an API, you can use the DataSource API to connect the dtSearch Engine to the cloud storage service for indexing.  For more information on this option, please see:

How to index databases with the dtSearch Engine
API Overview -- Indexing Databases
.NET DataSource API documentation
.NET Standard/.NET Core DataSource API documentation