Using the Studio Data SDK, you can manage your dataset records with several data endpoints.

---

# Upload Dataset

Create a [dataset](https://docs.foursquare.com/developer/docs/studio-data-sdk-classes#dataset) from a file upload.

### Python
```python
data_sdk.upload_file(
    file='new_file.csv',
    name='Dataset name',
    media_type=MediaType.CSV,
    description='Dataset description')
```

### CLI
```shell
fsq-data-sdk upload-file \
  --name "Dataset name" \
  --desc "Dataset description" \
  --media-type text/csv \
  new_file.csv
```

### HTTP
```http
curl -X POST https://data-api.foursquare.com/v1/datasets/data?name=My+Dataset \
-H 'Authorization: Bearer <token>' \
-H 'Content-Type: text/csv' \
--data-binary '@/path/to/my_dataset.csv'
```

# Upload Dataframe

You can use the Python SDK package to upload a Pandas or GeoPandas dataframe.

### Python
```python
data_sdk.upload_dataframe(
    dataframe,
    name='Dataset name',
    description='Dataset description')
```

# Create External Dataset

Create an external [dataset](https://docs.foursquare.com/developer/docs/studio-data-sdk-classes#dataset) record referencing a dataset by URL. External datasets are loaded from source every time, and _will not_ be stored in our system.

### Python
```python
data_sdk.create_external_dataset(
  name = "test-external-dataset",
  description = "my external dataset",
  source = "https://s3data.example.com/data-source",
  connector = "<data-connector-uuid>"
)
```

### CLI
```shell
fsq-data-sdk create-external-dataset \
  --source "test-external-dataset" \
  --name "my external dataset", \
  --description "https://s3data.example.com/data-source" \
  --connector-id "<data-connector-uuid>"
```

### HTTP
```http
curl POST 'https://data-api.foursquare.com/catalog/v1/datasets' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <TOKEN>' \
--data
'{
    "name": "My S3 Dataset",
    "type": "externally-hosted",
    "metadata": {
      "source": "s3://my-bucket/path/to/data.parquet"
    },
  "dataConnectionId": "<SOME_ID>"
}'
```

# Generate Vector Tiles

Create [Vector Tiles](https://docs.foursquare.com/studio/docs/data-formats-vector-tiles) by specifying a source GeoJSON (.geojson), CSV `.csv` or FlatGeobuf (`.fgb`) file, and optionally, a target dataset.

### Python
```python
data_sdk.generate_vectortile(
    source="source_dataset_uuid",
    target=None
)
```

### CLI
```shell
fsq-data-sdk generate-vectortile \
  --source "source-dataset-uuid" \
  --target "optional-target-uuid"
```

### HTTP
```http
curl --request POST \
     --url https://data-api.foursquare.com/v1/datasets/vectortile \
     --header 'accept: application/json' \
     --header 'content-type: application/json' \
     --data '
{
  "source": "source-dataset-uuid",
  "target": "target-dataset-uuid"
}
'
```

# Get Dataset Metadata

Get a [dataset record](https://docs.foursquare.com/developer/docs/studio-data-sdk-classes#dataset). This dataset record only includes the dataset's metadata—not the data itself. Pass the UUID or `dataset` object to receive the dataset's record as a JSON object.

### Python
```python
# Get dataset metadata by UUID
data_sdk.get_dataset_by_id("<uuid>")

# Get dataset record by dataset object
## List datasets
datasets = data_sdk.list_datasets()
dataset = datasets[0]
## Get dataset record by dataset object
data_sdk.get_dataset_by_id(dataset)
```

### CLI
```shell
fsq-data-sdk get-map <uuid>
```

### HTTP
```http
curl -X GET https://data-api.foursquare.com/v1/datasets/<uuid> \
  -H 'Authorization: Bearer <token>'
```

# Download Dataset

Download data for a given dataset. Pass the UUID or `dataset` object to get to receive the dataset. Provide `output_file` to write the dataset data to, or leave empty to return a `bytes` object with the dataset data.

### Python
```python
# Download dataset record by dataset object
## List datasets
datasets = data_sdk.list_datasets()
dataset = datasets[0]

# Download to local file
data_sdk.download_dataset(dataset, output_file='output.csv')
# Download to buffer
buffer = data_sdk.download_dataset(dataset)
```

### CLI
```shell
fsq-data-sdk download-dataset --dataset-id <uuid> --output-file output.csv
```

### HTTP
```http
curl -X GET https://data-api.foursquare.com/v1/datasets/<uuid>/data \
  -H 'Authorization: Bearer <token>'
```

### Python DataFrame
```python
# Download dataset record by dataset object
datasets = data_sdk.list_datasets()
dataset = datasets[0]

# Download to a dataframe
df = data_sdk.download_dataframe(dataset)
```

# Update Dataset

Update an existing dataset with a binary data upload. Can also update the name or description of the dataset. Pass the UUID or `dataset` object of the dataset to update.

### Python
```python
# Select a dataset to update
dataset = datasets[0]

# Update the dataset using file upload
data_sdk.update_dataset(
    dataset,
    name='New name'
    description='New description'
    file='new_file.csv',
    media_type=MediaType.CSV)
```

### CLI
```shell
fsq-data-sdk update-dataset --dataset-id <id> --media-type <media_type> --file <path> --name <name> --description <description>
```

### HTTP
```http
curl -X PUT https://data-api.foursquare.com/v1/datasets/<uuid>/data HTTP/1.1 \
    -H 'Authorization: Bearer <token>'\
    -H 'Content-Type: text/csv' \
--data-binary '@/path/to/my_dataset.csv'

curl --request PUT \
     --url https://data-api.foursquare.com/v1//v1/datasets/<uuid> \
     --header 'accept: application/json' \
     --header 'content-type: application/json' \
     --data '
{
  "name": "New name",
  "description": "New description"
}
'
```

# Delete Dataset

Delete a dataset record. Pass the UUID of the dataset to delete the dataset record. This will delete all data associated with the dataset.

### Warning!
>This operation cannot be undone. If you delete a dataset that appears in maps, the dataset will be removed from the map. This may cause the map to render incorrectly.

### Python
```python
# Delete dataset by dataset object

## Select dataset
datasets = data_sdk.list_datasets()
dataset = datasets[0]
## Delete selected Dataset
data_sdk.delete_dataset(dataset)

# Delete dataset by UUID
data_sdk.delete_dataset("<UUID>")
```

### CLI
```shell
fsq-data-sdk delete-dataset --dataset-id <uuid>
```

### HTTP
```http
curl -X DELETE https://data-api.foursquare.com/v1/datasets/<uuid> HTTP/1.1 \
    -H 'Authorization: Bearer <token>'
```

# List Datasets

List all [dataset](https://docs.foursquare.com/developer/docs/studio-data-sdk-classes#dataset) records on the authorized account.

### Python
```python
datasets = data_sdk.list_datasets()
```

### CLI
```shell
fsq-data-sdk list-datasets
```

### HTTP
```http
curl -X GET https://data-api.foursquare.com/v1/datasets HTTP/1.1 \
    -H 'Authorization: Bearer <token>'
```

### Python for Organization
```python
datasets = data_sdk.list_datasets(organization=True)
```

### CLI for Organization
```shell
fsq-data-sdk list-datasets --organization
```

### HTTP for Organization
```http
curl -X GET https://data-api.foursquare.com/v1/datasets/for-organization HTTP/1.1 \
    -H 'Authorization: Bearer <token>'
```

# Query Functions

A subset of the Data SDKs data functions, these endpoints allow users to query data from databases and data lakes added to Studio via [Data Connectors](https://docs.foursquare.com/studio/docs/connectors).

## List Data Connectors

List all [data connectors](https://docs.foursquare.com/developer/docs/studio-data-sdk-classes#data-connectors) added to the authorized account.

| Parameter | Type | Description |
| --- | --- | --- |
| `organization` | `boolean` | If `True`, list data connectors for organization of authenticated user. |

### Python
```python
# List data connectors associated with user account
data_connectors = data_sdk.list_data_connectors()
```

### CLI
```shell
fsq-data-sdk list-data-connectors
```

### HTTP
```http
curl -X GET https://data-api.foursquare.com/v1/data-connections HTTP/1.1
```

### Python for Organization
```python
# List data connectors associated with organization
data_connectors = data_sdk.list_data_connectors(organization=True)
```

### CLI for Organization
```shell
fsq-data-sdk list-data-connectors --organization
```

### HTTP for Organization
```http
curl -X GET https://data-api.foursquare.com/v1/data-connections/for-organization HTTP/1.1
```

## Execute Query

Execute a query against a data connector, returning a dataframe with the results of the query, or `None` of the output was written to a file.

| Parameter | Description |
| --- | --- |
| `connector` | Required. The data connector to use, or its UUID. |
| `query` | Required. The SQL query. |
| `output_file` | The path to write the query output to. |
| `output_format` | The format in which to write the output. |

### Python
```python
 df = data_sdk.execute_query(
    example_data_connector.id,
    "select * from table;"
)
```

### CLI
```shell
fsq-data-sdk execute-query --connector-id <connector uuid> --query <SQL query to use>
```

### HTTP
```http
curl -X POST https://data-api.foursquare.com/v1/query/gateway/data-queries HTTP/1.1
```

## Create Dataset from Query

Create a dataset from a query.

| Parameter | Description |
| --- | --- |
| `connector` | Required. The data connector to use, or its UUID. |
| `query` | Required. The SQL query. |
| `name` | Name of the dataset record. |
| `description` | Description for the dataset record. |

### Python
```python
create_query_dataset(query_dataset.id, "select * from table;", "query-dataset", "sample-description")
```

### CLI
```shell
fsq-data-sdk create-query-dataset --connector-id <connector uuid> --query <SQL query to use> --name <name for the new queried dataset> --description <description of the queried dataset>
```

### HTTP
```http
curl -X POST https://data-api.foursquare.com/v1/datasets/data-query HTTP/1.1
```