Driving data excellence: Qubika at the Austin Databricks Data Intelligence Day

This July, we joined Databricks’ “Data Intelligence Day” in Austin. The event was an opportunity to explore the capabilities of the Databricks unified Data Intelligence Platform and how industry leaders are using it to transform their organizations.

As a Databricks Select Partner with more than 150 certified engineers, we bring deep expertise in building and delivering scalable, high-impact data solutions. Our participation also builds on our close partnership with Databricks – following on from our Databricks meetups in Austin and Montevideo, as well as our SVP of Data & AI, Sebastian Diaz, delivering a session at the Data + AI Summit in San Francisco, on building scalable, high-value AI agents using Databricks and LangChain.

Unpacking the Databricks Data Intelligence Platform

A central theme of the event in Austin was analyzing the capabilities of the Databricks Data Intelligence Platform – a unified ecosystem designed to tackle the pervasive complexity often seen in modern data architectures. This platform seamlessly integrates crucial components:

• Mosaic AI for artificial intelligence: Driving the next generation of AI applications.
• DB SQL for data warehousing: Providing robust data analytics capabilities.
• Lakehouse for transactional database: Combining the best of data lakes and data warehouses.
• Lakeflow for ingest, ETL, and streaming: Streamlining data operations from raw ingestion to production workloads.
• Apps for secure data & AI applications: Enabling secure and scalable application development.
• Marketplace for data & AI exchange: Facilitating secure data and AI sharing.
• Underpinning this powerful platform are foundational technologies like Unity Catalog, Delta Lake, and Iceberg, ensuring data governance, reliability, and open standards.

Driving efficiency with Lakeflow and Agent Bricks

We were particularly impressed by the emphasis on boosting productivity and simplifying data workflows:

Lakeflow: Introduced only last year, Lakeflow is quickly becoming a game-changer for unified data operations. It’s empowing Databricks users building and operating data pipelines with:

• Easier data ingestion.
• Efficient and robust data transformation.
• Orchestrating production workloads with ease.

Agent Bricks: A new and exciting offering (announced during the 2025 DAIS) for building AI agents. We explored use cases such as “Information Extraction,” “Knowledge Factories,” “Multi-Agent Supervision,” and “Custom Logic,” all designed to automate and enhance business processes, complementing our expertise in Advanced AI and Machine Learning with Databricks, including using AI/BI Genie and Agent Bricks.

• Mosaic AI: The Databricks team showcased the latest developments of its Mosaic AI platform (a comprehensive platform for building sophisticated agent systems). Mosaic offers a structured approach covering the entire agent lifecycle:Prepare Data: From data ingestion and AI features to vector indexing.
• Build Agents: Leveraging GenAI models, classical ML models, and custom functions and tools.
• Evaluate Agents: Utilizing LLM Judges, peer labeling, and robust tracking for performance assessment.
• Deploy Agents: Through efficient agent serving, MLOps/LLMOps, and lineage tracking.
• Govern Agents: Ensuring responsible AI with guardrails, credential management, usage tracking, and rate limits.

This holistic approach helps you deliver accurate, domain-specific results, support both existing and future AI models, and build trust through rigorous evaluation and monitoring.

Our Data and AI Studios specialize in these areas, particularly in building enterprise-grade AI agents and copilots using the Databricks Mosaic AI framework, Agent Bricks, and our own Qubika Agentic Platform (QAP). We leverage tools like LangChain,Lakeflow, and MLflow to orchestrate complex agent pipelines, integrating retrieval-augmented generation (RAG), external APIs, and decision-making logic.

Boosting productivity for everyone – from data scientists to business users

The Databricks platform is designed to enhance productivity across the board, benefiting:

• Data Scientists: With intelligent assistance for complex tasks.
• Data Engineers: Streamlining metadata management and search.
• SQL Analysts: Providing intuitive tools for data exploration.
• Business Users: Through insightful AI/BI Dashboards and the innovative AI/BI Genie, making data-driven decisions more accessible.

This directly aligns with Qubika’s goal to deliver accelerated insights with faster ETL, real-time analytics, and deeper AI-driven insights.

Addressing the “complexity nightmare”

A key takeaway was how Databricks directly addresses the “complexity nightmare of high costs and proprietary formats” often associated with fragmented data landscapes. By unifying streaming, machine learning, generative AI, governance, orchestration, data lakes, data warehouses, and BI into a single platform, Databricks offers a powerful solution to this pervasive challenge, leading to improved scalability & reliability by eliminating siloed pipelines, reducing downtime, and enabling reprocessing for clean, auditable data.