This talk entitled "SQL on BigData - Architecture, Technology and Innovations" gives an exhaustive overview of landscape of SQL Engines on Big Data with focus on architectures of low latency SQL Engines. The talk covers SQL Engines in the open source space for Batch, Interactive, Streaming and Operational workloads. It discusses internals of tools like - Hive on Tez, Spark SQL, Impala, Apache Drill, Presto, Alluxio based architecture. SQL has been with us for over 40 years and Big Data about 10 years. Both of these technologies are here to stay and even though when Big Data was born there was no SQL interface to it - however with the rapid adoption of Big Data in the enterprise - the open source community and commercial organizations have built innovative solutions to build SQL Engines solutions on Big Data. The talk covers how SQL Engines are architected for processing Structured, UnStructured and Streaming Data the concepts behind them. The final portions of the talk covers the rapidly evolving landscape and innovations happening in the space - with products like OLAP on Big Data, Probabilistic SQL Engines like BlinkDB, HTAP based solutions like NuoDB and exciting solutions of using GPUs with 40000 cores to build massively massively parallel SQL Engines for large scale data sets with low latency. Benchmarking SQL Engines on Big Data has always been controversial and most of the Engines have used benchmark queries based on Relational DB whose architecture and workloads and capabilities are different. Final few slides of the talk covers TPC-Benchamrk 2.0 for evaluating performance of SQL Engines on Big Data. ======== Title - SQL on Big Data - Technology, Archicture and Innovations Video URL https://youtu.be/jUeSguiiYiQ ========= Basic Big Data Knowledge Basic Knowledge of how Database Engines work Basic interest in what is happening in the world of SQL on Big Data ======= Takeaway for Audience This talk is for both beginners and intermediate level Big Data Developers, Business Analysts and Database Engineers who would love to explore the internal architectures of SQL Engines on Big Data - how they are built, what kind of workloads they support - from Batch, Interactive, Real Time workloads to supporting SQL on Unstructured Data and how SQL Engines On Big Data sets are also being used for Operational Analytics and reporting. ======= Proposal With the rapid adoption of Big Data in the enterprise it has become all the more important to build SQL Engines on Big Data for all kinds of workloads for almost all kind of end users and use cases. From low latency analytics based SQL to ACID based semantics on Big Data for Operational Systems, to SQL for handling unstructured and streaming data, SQL is fast becoming the ligua-franca in the big data world too. The talk focuses on the exciting tools, technologies and innovations and their underlying architectures and the exciting road ahead in this space. This is a fiercely competitive landscape with vendors and innovators trying to capture mindshare and piece of the pie – with a whole suite of innovations like – index based SQL solutions in Big Data to OLAP with Apache Kylin and Tajo to BlinkDB and MapD. Topics : - Why SQL on Big Data - Challenges of SQL on Big Data - SQL on Big Data Architectures for Low Latency Analytics ( Drill, Impala, Presto, SparkSQL, JethroData) - SQL on Big Data Architecture for Semi-Structured Data - SQL on Big Data Architecture for Streaming Data - Innovations ( OLAP on Big Data, Probabilistic SQL Engines, GPU Based SQL Solutions ) ==== Description This presentation gives an exhaustive overview of the architectural and technological underpinnings of SQL on Big Data solutions of today. It covers architectures of low latency SQL engines on Big Data for structured, unstructured and streaming analytics as well as SQL for Operational Systems and Operational Analytics on Big Data. The talk also covers the innovations happening in the space with probabilistic engines like BlinkDB to GPU based engines like MapD. ===== Additional Notes Audio Podcast of this talk - as given in Apache BigData Conference in Vancouver ( May 10th, 2016) is located here http://feathercast.apache.org/?s=sumit This talk has been presented at 4 places - Apache Big Data Conference in Vancouver - May 10th, 2016 - Word Data Center Conference - 18th March, 2016, Las Vegas, - Big Data Tech Conf - 4th Nov, 20115 - Chicago - Global Big Data Conference - 11th Aug, 2015 - Boston