Justin Miller

OpenSearch vs LanceDB for Vector Search: Performance and Cost

justinrmiller@gmail.com (Justin Miller) — Sun, 12 Apr 2026 12:00:00 -0700

Choosing a vector database usually comes down to a tradeoff between a full search service and an in-process library. OpenSearch and LanceDB sit on opposite ends of that spectrum: one runs as a distributed cluster, the other as an embedded file format you query directly from your application. This post benchmarks both on the same workload (287,360 COCO 2017 images embedded with SigLIP 2) measuring ingestion throughput, query latency, storage layout, and AWS cost.

Building an E2E Encrypted Chat Application with LanceDB and libsodium

justinrmiller@gmail.com (Justin Miller) — Sat, 28 Mar 2026 00:00:00 -0700

Building a chat application where the server never sees plaintext messages requires careful coordination between cryptographic primitives, real-time delivery, and persistent storage. This post examines Seal, an end-to-end encrypted chat application that pairs LanceDB for zero-infrastructure storage with libsodium for audited, high-performance cryptography. The goal is to make it as easy as possible to deploy and inexpensive to run through object storage.

Introduction

End-to-end encrypted messaging systems face a fundamental tension: the server must store and relay messages it cannot read. This creates challenges that don’t exist in traditional chat applications:

Building a Ray Data Integration for Apache Kafka with confluent-kafka

justinrmiller@gmail.com (Justin Miller) — Sat, 20 Dec 2025 12:00:00 -0700

Streaming data to Kafka from distributed data processing pipelines is a common pattern, but the integration details matter. This post examines a Ray Data sink for Kafka built on confluent-kafka, leveraging the high-performance librdkafka library for efficient distributed writes.

The Problem

Writing Ray Datasets to Kafka presents several challenges that don’t align cleanly with typical batch processing patterns:

Converting DataFrames to Kafka message formats (key-value pairs)
Choosing appropriate serialization strategies (JSON, strings, bytes)
Managing asynchronous delivery and callbacks
Handling producer buffer overflow in high-throughput scenarios
Batching efficiently without blocking on network I/O
Managing message keys for partitioning and compaction

Without careful design, you end up with inefficient writes, memory issues, or silent data loss.

Building Ray Data Integrations for Vector Databases

justinrmiller@gmail.com (Justin Miller) — Tue, 08 Jul 2025 12:00:00 -0700

Vector databases like Turbopuffer are becoming essential for AI applications, but integrating them can be a challenge if careful considerations around batching and performance aren’t made. This post examines an integration that handles the details of distributed vector operations.

The Problem

Vector databases need specific data formats and batch strategies that may not align well with DataFrame operations. You end up dealing with:

Converting between DataFrames and vector formats
Optimizing batch sizes for network efficiency
Handling failures in distributed environments
Managing vector arrays and metadata together

Architecture: Datasource and Datasink

The Ray-Turbopuffer integration implements custom TurbopufferDatasource and TurbopufferDatasink classes that handle bidirectional data flow:

Interfacing Ray with internal libraries and services using Pydantic

justinrmiller@gmail.com (Justin Miller) — Wed, 11 Jun 2025 12:00:00 -0700

Companies often develop internal libraries that expose inputs and outputs to functions via Pydantic. Whether your internal library is for machine learning, API clients, or data validation services, you’ll often need to convert large DataFrames into Pydantic objects and pass batches of them to libraries.

Here’s how to do it efficiently using Ray Data.

The Challenge

Many in-house Python libraries/services follow these patterns:

ml_service.predict_batch(records: List[PredictionRequest]) -> List[PredictionResponse]
validator.validate_batch(data: List[DataModel]) -> List[DataValidation]
api_client.submit_batch(objects: List[APIRecord]) -> List[APIResponses]

But your data starts as DataFrames. This approach doesn’t scale if there’s any latency in the sequential processing of a batch:

Generating dbt Documentation with OpenAI and GitHub Actions

justinrmiller@gmail.com (Justin Miller) — Mon, 21 Apr 2025 12:00:00 -0700

In this blog post, I’ll walk through how to automatically generate documentation for your dbt project using OpenAI—completely integrated into your GitHub Actions pipeline. This process will analyze every dbt model and seed file and generate detailed DBT documentation in Markdown with OpenAI’s GPT 4o LLM that’s committed back to the pull request.

Technology Stack

dbt: The analytics engineering tool for transforming data in your warehouse.
OpenAI GPT4o (via API): Used to infer and document dbt models with markdown output.
GitHub Actions: Automates the entire flow on every pull request.
Typer CLI: Simplifies local development of the script used to collect and submit prompt data.

Advantages of This Approach

Automated Metadata Generation: Removes the burden of writing and maintaining technical documentation.
Contextual, Business-Friendly Summaries: The model extracts intent, joins, lineage, and structure.
Seamless GitHub Integration: Keeps docs in sync by auto-committing the output to the PR branch.

Prerequisites

Before running the workflow, make sure the following are in place:

LLM Inference with the Airflow AI SDK and Ollama

justinrmiller@gmail.com (Justin Miller) — Sun, 06 Apr 2025 12:00:00 -0700

In this blog post, I will demonstrate how the Airflow AI SDK and Ollama can be used to develop Airflow DAGs at no cost locally with Astro CLI. This allows for full end-to-end testing of the Airflow DAG without reaching out to a third-party LLM provider and for a broader range of model selection. The example takes a collection of product names, submits to Ollama a request with a system prompt designed to generate blog ideas, then prints out the blog ideas.

Fine Tuning Gemma 3 with Unsloth

justinrmiller@gmail.com (Justin Miller) — Sun, 23 Mar 2025 12:00:00 -0700

In this post, I’ll walk through fine-tuning Gemma-3 using Hugging Face and the Unsloth library. This article is based off an Unsloth Colab notebook and the consolidated source code is available here: Gemma 3 Training with Unsloth

To fully use the provided source code (for model export to HF) you must be logged into Huggingface via huggingface-cli.

Technologies Used

Hugging Face: Platform for hosting and training LLMs.
Unsloth: Library optimized for fast, efficient fine-tuning.
TRL (Transformer Reinforcement Learning): Provides SFTTrainer for supervised fine-tuning.
PyTorch: Core framework for model computations.

Steps Involved

The fine-tuning process involves several key steps:

Automating Ray Workflows with Astronomer and Ray on Vertex AI

justinrmiller@gmail.com (Justin Miller) — Sun, 08 Dec 2024 12:00:00 -0700

Introduction

Orchestrating distributed workloads efficiently is essential to maintaining performance and cost effectiveness of modern data engineering and machine learning workflows. By combining Astronomer, Ray, and Google Cloud Vertex AI, you can dynamically create and manage Ray clusters to execute jobs and clean up resources afterward. In this post, we’ll work through a process to automate these steps.

Why Astronomer, Ray, and Vertex AI?

Astronomer provides a managed Apache Airflow platform to orchestrate complex workflows.
Ray enables distributed computing for machine learning, deep learning, and data-intensive applications.
Vertex AI offers managed services for scalable and cost-effective resource management on Google Cloud.

Benefits of Using Astronomer with Ray on Vertex AI

Dynamic Ray Cluster Configuration: Spin up Ray clusters that are sized to the needs of the Ray job with data retrieved from sources like Snowflake or Redshift.
Cost Control: Spin down resources whenever the job completes to save on cluster resource costs.
Simplified Local Development: Easily set up and test workflows locally before deploying to production.
Effortless Deployment: Astronomer CLI provides one command to push updates to the Astronomer platform.
Version Control: Seamlessly integrates with Git for managing your DAGs and dependencies.

Pre-requisites

Before you begin, ensure the following:

Using LLMs and Vector Stores to search the Berkshire Hathaway Letters

justinrmiller@gmail.com (Justin Miller) — Mon, 14 Aug 2023 12:00:00 -0700

Extracting information from text can be a time consuming task, especially when you’re looking for a needle in a haystack. Warren Buffett has had a long and successful (perhaps the most successful?) career in investing and provides a number of anecdotes and words of wisdom in the form the Berkshire Hathaway shareholder letters. In this blog post I will lay out a method of extracting this wisdom via embeddings, a vector store, a large language model and a search interface via Streamlit.

CockroachDB Introduction

justinrmiller@gmail.com (Justin Miller) — Thu, 08 Jun 2017 12:00:00 -0700

This blog post is a combination of notes and references used in my talk on June 8th, 2017 at ProtectWise.

SQL
ACID transactions but also scalable, consistent, and HA
Raft consensus
Features both Linearizability and Serializability but without atomic clocks (Spanner)
Currently supported languages:
- C++
- Clojure
- Go
- Java
- Node.JS
- PHP
- Python
- Ruby
- Rust
Aphyr’s Review

Definitions

Linearizability - absolutely ordered events
Serializability - guarantees that the constituent reads and writes within a transaction occur as though that transaction were given exclusive access to the database for the length of its execution, guaranteeing that no transactions interfere with each other

Scala Collections Series, Part 3 - Operations on Scala Collections, File I/O and Trivia

justinrmiller@gmail.com (Justin Miller) — Sun, 28 May 2017 12:00:00 -0700

This blog post is the last of a three part series, this section covers:

Operations on Scala Collections
File I/O
Trivia

Operations on Scala Collections

It’s possible to compose operations on collections, here are a few examples.

scala> val listOfSentences = List("'Tis better to have loved and lost than never to have loved at all. #lovedthanlost", "It ain't what you don't know that gets you into trouble, it's what you know for sure that just ain't so. #marktwain", "#moon #onesmallstepforman That's one small step for man, one giant leap for mankind.")
listOfSentences: List[String] = List('Tis better to have loved and lost than never to have loved at all. #lovedthanlost, It ain't what you don't know that gets you into trouble, it's what you know for sure that just ain't so. #marktwain, #moon #onesmallstepforman That's one small step for man, one giant leap for mankind.)

scala> val numberOfWords = listOfSentences.flatMap(_.split(" ")).size
numberOfWords: Int = 50

scala> val distinctWords = listOfSentences.map(_.filter(c => !Set(',', '.', '!', ',').contains(c))).flatMap(_.split(" ")).toSet
distinctWords: scala.collection.immutable.Set[String] = Set(it's, trouble, for, That's, giant, have, #lovedthanlost, step, than, mankind, sure, what, so, #moon, all, just, #marktwain, ain't, man, don't, lost, that, to, you, know, small, 'Tis, at, leap, #onesmallstepforman, loved, gets, It, into, better, and, one, never)

scala> distinctWords.size
res102: Int = 38

scala> val hashtags = listOfSentences.map(_.filter(c => !Set(',', '.', '!', ',').contains(c))).flatMap(_.split(" ")).filter(_.head == '#')
hashtags: List[String] = List(#lovedthanlost, #marktwain, #moon, #onesmallstepforman)

File I/O

Input file:

Scala Collections Series, Part 2 - More Fundamental Structures

justinrmiller@gmail.com (Justin Miller) — Fri, 26 May 2017 12:00:00 -0700

This blog post is the middle of a three part series, this section covers:

Sets
Maps (HashMap and TreeMap)
Array and List Buffers
Queues and Stacks

For performance characteristics of the various data structures listed below, please see: Performance Characteristics

Sets

Sets in Scala are relatively straightforward. There are few interesting bits of trivia around Sets which will be covered in the trivia section.

For now, here are a number of examples of their use:

Scala Collections Series, Part 1 - Mutability and Lists

justinrmiller@gmail.com (Justin Miller) — Wed, 24 May 2017 12:00:00 -0700

This blog post is the beginning of a three part series that will cover a variety of topics around Scala Collections, this part will cover:

Mutable and Immutable Collections
Lists
Traversable and Iterable
Sequence Traits (Seq, IndexedSeq, LinearSeq)

Mutable and Immutable Collections

Collections packaged in the Scala Collections library can have mutable and immutable variants, generally separated by package name. This can make switching between the two very straightforward, but may require aliasing them if you use immutable and mutable data structures in the same file.

Scala Days 2016

justinrmiller@gmail.com (Justin Miller) — Mon, 20 Jun 2016 12:00:00 -0700

This year I wasn’t able to make it to Scala Days (hoping next year I’ll get to go!). Quite a few of the talks though seemed really interesting and luckily the Scala Days talks were made available shortly after the conference. Here’s my notes from a few of the videos (I’ll add more as I have time).

First, here’s a link to all the videos: Scala Days NY 2016

Martin Odersky, Keynote: Scala’s Road Ahead
- Slide deck: slides
- Scala Center (2:15)
  - a foundation supported by Lightbend, IBM, etc. with a focus on projects for the Scala community
- Scala 2.12 (3:00)
  - Targeted for mid-2016 release
  - Uses Java 8 lambdas/default methods for better performance
- Programming in Scala v3 is out and it’s updated for 2.12
- Scala 2.13 (5:05)
  - Focus on the libraries
  - Simpler libraries/in line Spark usage/lazy collections (views)
  - Potentially splitting Scala standard library into core and platform
- Scala.js (10:00)
  - Version 0.6.9
  - Anonymous classes/JUnit support/new site
- DOT (11:00)
  - Proven to be a sound foundation for Scala
  - A minimal language subset about which formal statements can be made and proven
  - DOT Terms (13:00) - he covers the Scala notation covered by DOT (slide 17)
  - Covers the soundness of the language
- dotty (16:25)
  - Working name for new Scala compiler
  - Builds on DOT in it’s internal data structures, but the language will still provide higher level language features (such as generics)
  - About twice the speed of nsc (current scala compiler) and should improve significantly in the future
- Dropped features (28:00)
  - Procedure Syntax ( for instance: def someFun(x:String) { x.length } )
  - Macros
  - Early initializers
  - Existential Types
- New features (32:30)
  - Intersection/Union Types
  - Named type parameters
  - Non-blocking lazy vals
  - Trait parameters
  - @static methods and fields
- Improvements in detail (39:30)
- Read this:
  - Strategic Scala Style: Principle of Least Power
Tim Soethout, Implicits Inspected and Explained

Category Theory for Programmers

justinrmiller@gmail.com (Justin Miller) — Mon, 23 May 2016 12:00:00 -0700

Lately I’ve become interested in category theory and it’s application to software engineering. After watching a video titled “Programming isn’t Math” by Oscar Boykin, I decided to spend a bit of time watching videos on libraries out of Twitter and conferences around category theory, Algebird, Storehaus and other libraries. Here’s a list of a few videos along with a brief summary of each.

Oscar Boykin, Programming Isn’t Math
- generalizing a matrix library with monoids
- HyperLogLog as a monoid
  - cardinality with ~1% error for large cardinalities w/storage of ~16 KB
- at around 28:00 mentions int2BigEndian isn’t a true bijection, interesting example of a coded bijection that isn’t truly a bijection but still useful
- mentioned Spire
Ian O’Connell, Algebird: SF Scala @Twitter

Exporting and plotting GPS coordinates from cell phone photos

justinrmiller@gmail.com (Justin Miller) — Tue, 22 Dec 2015 12:00:00 -0700

Recently I was interested in seeing all the places I’ve taken photos and found a way of extracting exif data (specifically lat/long) from images stored on my cell phone. Here are the steps I used.

Generate the CSV file

To begin, start by installing exiftool. This can done on OS X by running “brew install exiftool” (assuming you have Homebrew installed).

The following will generate a CSV file (photogps.csv) from jpg images containing the name of the file, GPS lat, and GPS long.

FileVault 2 Performance Numbers

justinrmiller@gmail.com (Justin Miller) — Thu, 04 Jun 2015 12:00:00 -0700

A quick post to demonstrate the differences in performance when FileVault 2 is enabled vs disabled.

Today I re-installed OS X and ran various benchmarking tools (with and without FileVault 2 enabled). Here are the results:

FileVault 2 Enabled:

FileVault 2 Disabled:

As we can see from the numbers above (I ran them repeatedly to ensure there wasn’t much of a difference between runs), the performance penalty is not nearly as pronounced as one might expect. As a result I plan using full-disk encrypting both my laptop and Time Machine backup.

Justin Miller

OpenSearch vs LanceDB for Vector Search: Performance and Cost

Building an E2E Encrypted Chat Application with LanceDB and libsodium

Introduction

Building a Ray Data Integration for Apache Kafka with confluent-kafka

The Problem

Building Ray Data Integrations for Vector Databases

The Problem

Architecture: Datasource and Datasink

Interfacing Ray with internal libraries and services using Pydantic

The Challenge

Generating dbt Documentation with OpenAI and GitHub Actions

Technology Stack

Advantages of This Approach

Prerequisites

LLM Inference with the Airflow AI SDK and Ollama

Fine Tuning Gemma 3 with Unsloth

Technologies Used

Steps Involved

Automating Ray Workflows with Astronomer and Ray on Vertex AI

Why Astronomer, Ray, and Vertex AI?

Benefits of Using Astronomer with Ray on Vertex AI

Pre-requisites

Using LLMs and Vector Stores to search the Berkshire Hathaway Letters

CockroachDB Introduction

Definitions

Further reading

Scala Collections Series, Part 3 - Operations on Scala Collections, File I/O and Trivia

Operations on Scala Collections

File I/O

Scala Collections Series, Part 2 - More Fundamental Structures

Sets

Scala Collections Series, Part 1 - Mutability and Lists

Mutable and Immutable Collections

Scala Days 2016

Tim Soethout, Implicits Inspected and Explained

Category Theory for Programmers

Exporting and plotting GPS coordinates from cell phone photos

Generate the CSV file

FileVault 2 Performance Numbers