Accelerate big data analysis with the Spark 3.0 Compatible Connector for SQL Server – now in preview.
We announce that the preview version of Apache Spark 3.0 is compatible Apache Spark Connector for SQL Server and Azure SQL, available through Maven.
The Apache Spark Connector for SQL Server, released as open source in June 2020, is a powerful connector that allows you to use transactional data in big data analytics and retain results for ad hoc queries or reports. You can use SQL Server or Azure SQL as input data sources or output data sinks for Spark jobs. Previously, the connector supported Spark 2.4 workloads. Now you can use the connector to take advantage of the many advantages of Spark 3.0 as well.
Why should you use the Apache Spark Connector for SQL Server and Azure SQL?
The Apache Spark Connector for SQL Server and Azure SQL is based on the Apache Spark DataSourceV1 API and the SQL Server Bulk API and uses the same interface as the built-in Spark SQL Connector for Java Database Connectivity (JDBC). This way you can easily integrate the connector and migrate your existing Spark jobs simply by updating the format parameter.
Notable features and benefits of the connector:
• • Compatible with Apache Spark 3.0.
• Support for all Apache Spark bindings (Scala, Python, R).
• Basic authentication, Active Directory (AD) Keys tab, and Azure Active Directory support.
To learn more about the connector and how to use it, visit GitHub page. The 3.0 compatible port is available through the spark-mssql-connnector_2.12_3.0: 1.0.0-alpha Maven coordinate.
The Apache Spark Connector for SQL Server and Azure SQL makes the interaction between SQL Server and Apache Spark error-free. The connector has a growing and dedicated community and has been installed thousands of times. We are constantly developing and improving the connector. We look forward to your feedback and your contributions.
Note: The connector is supported by the community and does not offer Microsoft SLA support. Please report a problem on GitHub to ask the community for help.