Using dtSearch products with cloud storage (OneDrive, Amazon S3, etc.)

Article: dts0243

The same process of adding folders to index that works for localized content also works for cloud storage like OneDrive, DropBox and SharePoint-synced files that are visible through the Windows file system.

For example, DropBox will add a "DropBox" folder on your system and OneDrive will add a "OneDrive" folder on your system. To index these types of folders, click "Add Folder" in the Update Index dialog box. Typically this will add a folder named C:\Users\<your name>\DropBox or C:\Users\<your name>\OneDrive to the index, which is where this data is stored locally.

Some cloud storage services instead or alternatively provide access through a network share. For example, you can set up OneDrive for Business to be accessible through a network share. To index this type of data, add the network share to the index, again using "Add Folder". The network share will have a UNC format, like this: \\server\share\folder.

Amazon S3 does not create a local folder or network share by default, but there are several third-party products that can do this.

For SharePoint, please see: How to index SharePoint sites.

dtSearch Engine

(1) File system indexing

The options described above will also work with the dtSearch Engine.

(2) Direct integration with the cloud storage API.

The dtSearch Engine has a "DataSource" API that is designed to enable developers to connect to content that is accessible through an API. Most commonly this is used with databases but the API is completely general and can be used with any content. Each logical document indexed can consist any combination of plain or Unicode text, a set of field-value pairs, and a binary document file to be indexed using the dtSearch Engine's built-in document filters. The file can be provided as a disk file, memory buffer, or stream object.

If a cloud storage service has an API, you can use the DataSource API to connect the dtSearch Engine to the cloud storage service for indexing. For more information on this option, please see:

How to index databases with the dtSearch Engine

API Overview -- Indexing Databases

.NET DataSource API documentation

.NET Standard/.NET Core DataSource API documentation