Data
Data collections turn files and records from external systems into reusable, searchable knowledge for Siesta AI agents. This section explains every Data source currently available in the dev application and what belongs in each field.
The Data Model
- A connection stores authentication for an external system.
- A data collection is the access-controlled business container.
- A data source defines exactly what to import through a connection.
- Documents and chunks are the processed content that retrieval can search.
- An agent retrieves from one or more assigned collections.
Credentials belong in Connections. Folder IDs, paths, blob selectors, project or space keys, and URLs belong in the Data-source form.
Start Here
| Goal | Guide |
|---|---|
| Compare all sources and decide where content belongs | Choose a Data Source |
| Upload a controlled file snapshot | Manual Upload |
| Set the default or user-specific Data upload allowance | Data Upload Limits |
| Synchronize Google folders or Shared Drives | Google Drive |
| Synchronize OneDrive or SharePoint folders | Microsoft 365 |
| Read blobs or Azure file shares | Azure Storage |
| Automate a governed local or on-premises document feed | Automated File Ingestion with Azure File Share |
| Import Jira projects or Confluence spaces | Atlassian |
| Scrape one page or crawl a bounded website | Firecrawl |
| Configure processing, retrieval, sync, and diagnose failures | Processing, Sync, and Troubleshooting |
When To Use Data
Use a collection when several users or agents should reuse the same approved source material, content should synchronize, processing and logs must be inspectable, or the knowledge needs a private, team, or organization access boundary.
For a single temporary file, a chat attachment can be simpler. For durable knowledge, use a collection.
Create a Collection
Open Data and select Create collection.
- Name: use a durable business name such as
Customer Support — Approved Knowledge. - Description: record content scope, owner, exclusions, and intended agents.
- Visibility: choose one of three access scopes:
- Private: only you and people you explicitly grant access to can use the collection.
- Entire organization: all organization members receive the permission selected under Organization permission.
- Selected teams: only members of the teams added under Team Access receive access.
The collection is the access boundary. Do not mix confidential HR documents and public product documentation merely because both are stored in Google Drive.
Add A Data Source
Open a collection and select Add data source. Manual Upload is independent of Connections. Every integration-backed source first needs a compatible entry under Connections.
All connected-source forms share:
- Name identifying this specific import,
- optional Description with owner, scope, and exclusions,
- Connection ID selecting stored authentication,
- Sync frequency: On Demand, Daily, Weekly, or Monthly.
If Connection ID is empty, create or request access to the correct connection and return to the collection. Never paste a secret into this selector.
After clicking the desired data source, you configure it.
Enabling Advanced content extraction increases cost. It uses Azure Document Intelligence, billed per page. Enable it only for scanned or table-heavy documents.
Inspect Before Agents Use It
Check source status, representative files, Indexed and Readable state, extracted chunks, and Logs. A successful run proves that processing completed; it does not prove that every expected document is present, useful, current, or safe for the collection audience.