Using the Studio Data SDK, you can manage your dataset records with several data endpoints.

Upload Dataset

Create a dataset from a file upload.

Python

data_sdk.upload_file(
    file='new_file.csv',
    name='Dataset name',
    media_type=MediaType.CSV,
    description='Dataset description')

CLI

fsq-data-sdk upload-file \
  --name "Dataset name" \
  --desc "Dataset description" \
  --media-type text/csv \
  new_file.csv

HTTP

curl -X POST https://data-api.foursquare.com/v1/datasets/data?name=My+Dataset \
-H 'Authorization: Bearer <token>' \
-H 'Content-Type: text/csv' \
--data-binary '@/path/to/my_dataset.csv'

Upload Dataframe

You can use the Python SDK package to upload a Pandas or GeoPandas dataframe.

Python

data_sdk.upload_dataframe(
    dataframe,
    name='Dataset name',
    description='Dataset description')

Create External Dataset

Create an external dataset record referencing a dataset by URL. External datasets are loaded from source every time, and will not be stored in our system.

Python

data_sdk.create_external_dataset(
  name = "test-external-dataset",
  description = "my external dataset",
  source = "https://s3data.example.com/data-source",
  connector = "<data-connector-uuid>"
)

CLI

fsq-data-sdk create-external-dataset \
  --source "test-external-dataset" \
  --name "my external dataset", \
  --description "https://s3data.example.com/data-source" \
  --connector-id "<data-connector-uuid>"

HTTP

curl POST 'https://data-api.foursquare.com/catalog/v1/datasets' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <TOKEN>' \
--data
'{
    "name": "My S3 Dataset",
    "type": "externally-hosted",
    "metadata": {
      "source": "s3://my-bucket/path/to/data.parquet"
    },
  "dataConnectionId": "<SOME_ID>"
}'

Generate Vector Tiles

Create Vector Tiles by specifying a source GeoJSON (.geojson), CSV .csv or FlatGeobuf (.fgb) file, and optionally, a target dataset.

Python

data_sdk.generate_vectortile(
    source="source_dataset_uuid",
    target=None
)

CLI

fsq-data-sdk generate-vectortile \
  --source "source-dataset-uuid" \
  --target "optional-target-uuid"

HTTP

curl --request POST \
     --url https://data-api.foursquare.com/v1/datasets/vectortile \
     --header 'accept: application/json' \
     --header 'content-type: application/json' \
     --data '
{
  "source": "source-dataset-uuid",
  "target": "target-dataset-uuid"
}
'

Get Dataset Metadata

Get a dataset record. This dataset record only includes the dataset's metadata—not the data itself. Pass the UUID or dataset object to receive the dataset's record as a JSON object.

Python

# Get dataset metadata by UUID
data_sdk.get_dataset_by_id("<uuid>")

# Get dataset record by dataset object
## List datasets
datasets = data_sdk.list_datasets()
dataset = datasets[0]
## Get dataset record by dataset object
data_sdk.get_dataset_by_id(dataset)

CLI

fsq-data-sdk get-map <uuid>

HTTP

curl -X GET https://data-api.foursquare.com/v1/datasets/<uuid> \
  -H 'Authorization: Bearer <token>'

Download Dataset

Download data for a given dataset. Pass the UUID or dataset object to get to receive the dataset. Provide output_file to write the dataset data to, or leave empty to return a bytes object with the dataset data.

Python

# Download dataset record by dataset object
## List datasets
datasets = data_sdk.list_datasets()
dataset = datasets[0]

# Download to local file
data_sdk.download_dataset(dataset, output_file='output.csv')
# Download to buffer
buffer = data_sdk.download_dataset(dataset)

CLI

fsq-data-sdk download-dataset --dataset-id <uuid> --output-file output.csv

HTTP

curl -X GET https://data-api.foursquare.com/v1/datasets/<uuid>/data \
  -H 'Authorization: Bearer <token>'

Python DataFrame

# Download dataset record by dataset object
datasets = data_sdk.list_datasets()
dataset = datasets[0]

# Download to a dataframe
df = data_sdk.download_dataframe(dataset)

Update Dataset

Update an existing dataset with a binary data upload. Can also update the name or description of the dataset. Pass the UUID or dataset object of the dataset to update.

Python

# Select a dataset to update
dataset = datasets[0]

# Update the dataset using file upload
data_sdk.update_dataset(
    dataset,
    name='New name'
    description='New description'
    file='new_file.csv',
    media_type=MediaType.CSV)

CLI

fsq-data-sdk update-dataset --dataset-id <id> --media-type <media_type> --file <path> --name <name> --description <description>

HTTP

curl -X PUT https://data-api.foursquare.com/v1/datasets/<uuid>/data HTTP/1.1 \
    -H 'Authorization: Bearer <token>'\
    -H 'Content-Type: text/csv' \
--data-binary '@/path/to/my_dataset.csv'

curl --request PUT \
     --url https://data-api.foursquare.com/v1//v1/datasets/<uuid> \
     --header 'accept: application/json' \
     --header 'content-type: application/json' \
     --data '
{
  "name": "New name",
  "description": "New description"
}
'

Delete Dataset

Delete a dataset record. Pass the UUID of the dataset to delete the dataset record. This will delete all data associated with the dataset.

Warning!

This operation cannot be undone. If you delete a dataset that appears in maps, the dataset will be removed from the map. This may cause the map to render incorrectly.

Python

# Delete dataset by dataset object

## Select dataset
datasets = data_sdk.list_datasets()
dataset = datasets[0]
## Delete selected Dataset
data_sdk.delete_dataset(dataset)

# Delete dataset by UUID
data_sdk.delete_dataset("<UUID>")

CLI

fsq-data-sdk delete-dataset --dataset-id <uuid>

HTTP

curl -X DELETE https://data-api.foursquare.com/v1/datasets/<uuid> HTTP/1.1 \
    -H 'Authorization: Bearer <token>'

List Datasets

List all dataset records on the authorized account.

Python

datasets = data_sdk.list_datasets()

CLI

fsq-data-sdk list-datasets

HTTP

curl -X GET https://data-api.foursquare.com/v1/datasets HTTP/1.1 \
    -H 'Authorization: Bearer <token>'

Python for Organization

datasets = data_sdk.list_datasets(organization=True)

CLI for Organization

fsq-data-sdk list-datasets --organization

HTTP for Organization

curl -X GET https://data-api.foursquare.com/v1/datasets/for-organization HTTP/1.1 \
    -H 'Authorization: Bearer <token>'

Query Functions

A subset of the Data SDKs data functions, these endpoints allow users to query data from databases and data lakes added to Studio via Data Connectors.

List Data Connectors

List all data connectors added to the authorized account.

Python

# List data connectors associated with user account
data_connectors = data_sdk.list_data_connectors()

CLI

fsq-data-sdk list-data-connectors

HTTP

curl -X GET https://data-api.foursquare.com/v1/data-connections HTTP/1.1

Python for Organization

# List data connectors associated with organization
data_connectors = data_sdk.list_data_connectors(organization=True)

CLI for Organization

fsq-data-sdk list-data-connectors --organization

HTTP for Organization

curl -X GET https://data-api.foursquare.com/v1/data-connections/for-organization HTTP/1.1

Execute Query

Execute a query against a data connector, returning a dataframe with the results of the query, or None of the output was written to a file.

Python

 df = data_sdk.execute_query(
    example_data_connector.id,
    "select * from table;"
)

CLI

fsq-data-sdk execute-query --connector-id <connector uuid> --query <SQL query to use>

HTTP

curl -X POST https://data-api.foursquare.com/v1/query/gateway/data-queries HTTP/1.1

Create Dataset from Query

Create a dataset from a query.

Python

create_query_dataset(query_dataset.id, "select * from table;", "query-dataset", "sample-description")

CLI

fsq-data-sdk create-query-dataset --connector-id <connector uuid> --query <SQL query to use> --name <name for the new queried dataset> --description <description of the queried dataset>

HTTP

curl -X POST https://data-api.foursquare.com/v1/datasets/data-query HTTP/1.1