# Open Data Hub Documentation > APIs and data ingestion documentation for the Open Data Hub. - [Open Data Hub Documentation](/index.md) ## quickstart This guide follows the path most developers take: find the dataset you need, inspect it, then read it from the API. Most Open Data Hub data is open and needs no authentication. - [Quickstart: your first request](/quickstart.md): This guide follows the path most developers take: find the dataset you need, inspect it, then read it from the API. Most Open Data Hub data is open and needs no authentication. ## domains-and-datasets The Open Data Hub provides structured access to datasets across multiple domains through REST APIs. As an API user or ingestion creator, you interact with these datasets via standardized endpoints, metadata, and response formats. This guide covers the core concepts you need to navigate and retrieve data effectively. - [Domains and datasets](/domains-and-datasets.md): The Open Data Hub provides structured access to datasets across multiple domains through REST APIs. As an API user or ingestion creator, you interact with these datasets via standardized endpoints, metadata, and response formats. This guide covers the core concepts you need to navigate and retrieve data effectively. ## licensing The Open Data Hub provides open, freely accessible data under permissive licenses, ensuring compliance with open data principles. As an API user or ingestion creator, you can rely on structured license metadata to understand the terms under which data can be used. - [Data licensing](/licensing.md): The Open Data Hub provides open, freely accessible data under permissive licenses, ensuring compliance with open data principles. As an API user or ingestion creator, you can rely on structured license metadata to understand the terms under which data can be used. ## use-data ### authentication-and-access - [Authentication and authorization](/use-data/authentication-and-access/authentication.md): To access protected data in the Open Data Hub APIs, you must authenticate using OAuth 2.0 through Keycloak and include a valid access token in your API requests. This guide explains how to obtain and use tokens, manage their lifecycle, and request access to closed datasets. All instructions are written for API users who need to consume data from the Content API or Time Series API. - [Quotas, CLI and the R package](/use-data/authentication-and-access/quotas-and-tools.md): 1. API quotas and rate limiting ### content-api - [AlpineBits integration](/use-data/content-api/alpinebits.md): The Content API supports integration with the AlpineBits standard, enabling access to tourism data through standardized endpoints. Open Data Hub implements two key datasets defined by the AlpineBits Alliance: HotelData and DestinationData. These datasets are served via separate endpoints and require specific request formatting to retrieve data successfully. - [Deprecations and datamodel migration](/use-data/content-api/deprecations.md): 1. Introduction - [Filtering and sorting (Content API)](/use-data/content-api/filtering-and-sorting.md): The Content API provides a comprehensive set of query parameters to filter, sort, and shape the data returned from its endpoints. These tools allow API users to retrieve only the data they need, reduce payload size, and organize results effectively. This guide covers the core filtering and sorting capabilities available across Content API endpoints. - [Output formats (Content API)](/use-data/content-api/output-formats.md): 1. JSON and JSON-LD response formats - [Content API reference](/use-data/content-api/reference.md): The Content API provides access to tourism-related datasets such as accommodations, activities, events, weather, and more. It exposes a set of standardized endpoints for retrieving structured data, with consistent query parameters, response formats, and metadata across all entity types. - [Content API explorer](/use-data/content-api/swagger.md): Try the Content API directly in your browser below. You can also open the full Swagger UI on the API host, which is the most reliable place to run live requests. ### time-series-api - [Filtering time series data](/use-data/time-series-api/filtering.md): The Time Series API supports flexible filtering of mobility data using the where query parameter with dot-notation operators. This enables precise retrieval of station and time series data based on attribute values, spatial relationships, and logical combinations. Unlike SQL, the filter syntax uses a functional, dot-separated format that is both human-readable and URL-safe. - [Time Series API reference](/use-data/time-series-api/reference.md): The Time Series API provides access to mobility-related time series data through a unified endpoint structure. This reference describes the API's base structure, available representations, filtering capabilities, query parameters, and response format. - [Time Series API explorer](/use-data/time-series-api/swagger.md): Try the Time Series API directly in your browser below. You can also open the full Swagger UI, which is the most reliable place to run live requests. ### transmodel-api - [Transmodel API: NeTEx and SIRI-Lite](/use-data/transmodel-api/reference.md): The Transmodel API provides standardized access to both static and real-time mobility data in compliance with European and Italian interoperability standards. It supports two key formats: NeTEx for scheduled data and SIRI-Lite for real-time updates. This API is part of the MaaS4Italy initiative and integrates with the Open Data Hub, enabling third-party applications to retrieve mobility information through the National Access Point (NAP). - [Transmodel API explorer](/use-data/transmodel-api/swagger.md): Try the Transmodel API directly in your browser below. You can also open the full Swagger UI, which is the most reliable place to run live requests. ## data-ingestion ### collector-blueprints - [API Crawler Collector](/data-ingestion/collector-blueprints/api-crawler.md): The API Crawler collector is designed to pull data from external APIs based on a declarative configuration. It allows for complex and dynamic API interactions, including multi-step calls, data transformations, and context-based processing. This collector is ideal for scenarios where data needs to be fetched from various API endpoints, potentially requiring pagination, authentication, or nested requests. - [MQTT Client Collector](/data-ingestion/collector-blueprints/mqtt-client.md): A pre-built microservice designed to easily integrate with data sources that publish information via the MQTT protocol. - [Rest Push Collector](/data-ingestion/collector-blueprints/rest-push.md): The Rest Push collector acts as a generic endpoint where external data providers can push their data directly to the Open Data Hub. This means you, as a data provider, don't need to deploy any additional software or agents on your side. Instead, you'll be provided with specific credentials and a URL, and you can simply send your data via standard HTTP POST requests. - [S3 Poller Collector](/data-ingestion/collector-blueprints/s3-poller.md): Designed for efficiently collecting data from AWS S3 buckets. This collector is ideal for scenarios where your data provider regularly uploads files to an S3 bucket, and you need to automatically retrieve these files and feed their content into the Open Data Hub pipeline. ### data-collector-from-scratch Developing a robust and reliable data collector is a critical step in any data integration pipeline. This guide will walk you through the essential principles, architectural patterns, and practical implementation details for building a data collector in Go, using the provided S3 poller example as a reference. - [Developing a Data Collector from Scratch](/data-ingestion/data-collector-from-scratch.md): Developing a robust and reliable data collector is a critical step in any data integration pipeline. This guide will walk you through the essential principles, architectural patterns, and practical implementation details for building a data collector in Go, using the provided S3 poller example as a reference. ### data-transformer-from-scratch A data transformer is the bridge between the raw data a collector produces and the standardized entities the Open Data Hub serves. It consumes raw data events from the message queue, transforms them, and pushes the result to the Open Data Hub. Where it pushes depends on the kind of data: - [Developing a Data Transformer from Scratch](/data-ingestion/data-transformer-from-scratch.md): A data transformer is the bridge between the raw data a collector produces and the standardized entities the Open Data Hub serves. It consumes raw data events from the message queue, transforms them, and pushes the result to the Open Data Hub. Where it pushes depends on the kind of data: - [Pushing to the Content API](/data-ingestion/data-transformer-from-scratch/content.md): This page covers the write side of a transformer that produces structured content (announcements, points of interest, events, and similar), using the clib package of the opendatahub-go-sdk. For the shared transformer skeleton (the SDK listener, environment variables, containerization, and local run), see Developing a Data Transformer from Scratch. - [Pushing to the Timeseries Writer (BDP)](/data-ingestion/data-transformer-from-scratch/timeseries.md): This page covers the write side of a transformer that produces time series measurements, using the go-bdp-client SDK. For the shared transformer skeleton (the SDK listener, environment variables, containerization, and the local development workflow), see Developing a Data Transformer from Scratch. ### development - [Structure:](/data-ingestion/development/intro.md): A data integration pipeline consists of at least 2 microservices: ### getting-started This guide provides a comprehensive overview of setting up your local development environment for Open Data Hub data collectors and transformers, focusing on Go-based implementations. You'll learn how to leverage Docker Compose for infrastructure, interact with essential endpoints, handle authentication for the Timeseries Writer, and get hands-on with creating and testing a collector and a transformer. - [Getting Started](/data-ingestion/getting-started.md): This guide provides a comprehensive overview of setting up your local development environment for Open Data Hub data collectors and transformers, focusing on Go-based implementations. You'll learn how to leverage Docker Compose for infrastructure, interact with essential endpoints, handle authentication for the Timeseries Writer, and get hands-on with creating and testing a collector and a transformer. ### sdks The Open Data Hub provides Software Development Kits (SDKs) to streamline the development of data integration components, such as data collectors and transformers. These SDKs abstract away common concerns like messaging, telemetry, and interaction with the Timeseries Writer and Content APIs, allowing developers to focus on the core business logic of data acquisition and transformation. - [SDKs](/data-ingestion/sdks.md): The Open Data Hub provides Software Development Kits (SDKs) to streamline the development of data integration components, such as data collectors and transformers. These SDKs abstract away common concerns like messaging, telemetry, and interaction with the Timeseries Writer and Content APIs, allowing developers to focus on the core business logic of data acquisition and transformation. ### transformer-blueprints - [Transformer Boilerplate Generator](/data-ingestion/transformer-blueprints/transformer-boilerplate.md): As of now, the Open Data Hub does not offer a wide collection of pre-configured "Transformer Blueprints" in the same way as some Data Collectors. Instead, we provide a boilerplate generator that sets up a minimal, ready-to-extend Go-based transformer service. This allows you to quickly get started with a functional transformer and then customize its core transformation logic to fit your specific data requirements. ## tools ### analytics Open Data Hub Analytics is a web tool for visualizing Open Data Hub time series data without writing code. It is useful for quickly seeing how a measurement behaves over time, comparing stations, and sharing a view with others. - [Analytics](/tools/analytics.md): Open Data Hub Analytics is a web tool for visualizing Open Data Hub time series data without writing code. It is useful for quickly seeing how a measurement behaves over time, comparing stations, and sharing a view with others. ### data-browser The Open Data Hub Data Browser is a web-based tool for exploring, filtering, and managing Open Data Hub datasets through a user-friendly interface. For API users it is a convenient way to see the exact data model and fields the Content API returns; for content editors it is also where tourism content is created and maintained. - [Data Browser](/tools/data-browser.md): The Open Data Hub Data Browser is a web-based tool for exploring, filtering, and managing Open Data Hub datasets through a user-friendly interface. For API users it is a convenient way to see the exact data model and fields the Content API returns; for content editors it is also where tourism content is created and maintained. - [Managing content in the Data Browser](/tools/data-browser/content-management.md): This guide covers the editorial workflows in the Data Browser: maintaining events, images, tags, and news articles. It is aimed at content editors and authorized users. If you only need to read data through the APIs, see the Data Browser overview instead. ### discovery Open Data Hub Discovery is a web tool for exploring and understanding the datasets in the Open Data Hub before you query them. It automatically discovers datasets from the Metadata API, infers their schema, and computes field-level statistics, so you can find the right dataset and the right fields without writing any code. - [Discovery](/tools/discovery.md): Open Data Hub Discovery is a web tool for exploring and understanding the datasets in the Open Data Hub before you query them. It automatically discovers datasets from the Metadata API, infers their schema, and computes field-level statistics, so you can find the right dataset and the right fields without writing any code. ### imageresizer 1. Image Proxy Functionality - [Imageresizer](/tools/imageresizer.md): 1. Image Proxy Functionality ### web-components The Open Data Hub provides a collection of reusable Web Components for integrating real-time data into websites and applications. These components, covering domains such as tourism, weather, and mobility, are designed to be easily embedded and configured without requiring deep integration with the underlying APIs. - [Web Components](/tools/web-components.md): The Open Data Hub provides a collection of reusable Web Components for integrating real-time data into websites and applications. These components, covering domains such as tourism, weather, and mobility, are designed to be easily embedded and configured without requiring deep integration with the underlying APIs.