What is Apache Spark?
Apache Spark is an open-source engine for distributed data processing. It supports SQL and DataFrame workloads, Structured Streaming and machine-learning libraries on a chosen runtime. Spark is an execution engine rather than a managed hosting service or a complete ingestion catalogue.
Who is Apache Spark for?
Data engineering teams needing programmable distributed transformations on selected infrastructure.
Apache Spark features
The core capabilities to consider for your workflow.
- Distributed SQL and DataFrames
- Structured Streaming
- Python, SQL, Scala and Java workflows
- Machine-learning libraries and cluster execution
Apache Spark pricing
Free processing engine; runtime separate
Estimate the actual cluster or managed-runtime resources, job duration, storage and data transfer. Include orchestration and support where needed.
Calculate the cost of your quote →Free access & evaluation
The upstream software is free. Compute, storage, managed platforms and commercial support are separate.
Before you choose Apache Spark
A deployment needs compute, storage, dependency management and operational ownership.
A supported API or connector does not establish equal behaviour across every managed Spark distribution.
Explore Apache Spark in more detail
Take a closer look at the decisions behind this product.
Spark vs Dataflow: execution model and operating costs
Read the articleCompare Apache Spark
Apache Spark vs Google Cloud Dataflow
Compare Apache Spark and Google Cloud Dataflow for batch and streaming pipelines, programming models, runtime ownership, state recovery and the complete compute bill.
Read the comparison
dbt vs Apache Spark
Compare dbt and Apache Spark for analytics transformations, model dependencies, execution ownership, testing and the boundary between project tooling and processing compute.
Read the comparisonExplore the individual profiles
Apache Spark FAQs
Does Apache Spark have a free plan?
The upstream software is free. Compute, storage, managed platforms and commercial support are separate.
Who is Apache Spark for?
Data engineering teams needing programmable distributed transformations on selected infrastructure.
What should I compare before buying?
Check the tier that contains your essential features, confirm plan limits and compare the total quote with your own expected usage.