The official tooling for working with Lexicons in JavaScript projects has virtually not evolved since it was first released. While it serves the atproto codebase well, it was not designed with a "third party devs first" mindset. This post explores a few areas in which the process known as "lexgen" could be improved.

Generated Code Bundle Issues

The codegen process produces output that suffers from several bundle size problems, making it difficult to build efficient production applications.

Duplicated validation logic: Currently, the @atproto/lexicon package depends on Zod, a validation library used to validate input schemas during codegen. This creates a fundamental problem: both Lexicon schemas and Zod serve the same purpose of data validation, resulting in duplicated logic. This duplication impacts bundle size in two ways. For applications that don't need runtime validation of Lexicon schemas, Zod shouldn't need to be bundled at all since schema validation only happens during codegen. However, because of the dependency structure, any runtime using the result of codegen inherits this dependency unnecessarily. For applications that do need runtime validation, such as those validating both Lexicon schemas and network data like records from the Firehose, this duplication means shipping two separate validation systems with overlapping functionality.

Lack of tree shaking: The codegen process yields two artifacts that compound the bundle size problem: schemaDict, a large JavaScript object containing all lexicons used during codegen, and the namespace class structure, a big structure of nested classes that allow referencing the full tree of imported lexicons records and XRPC methods. Neither of these structures allows bundlers to "shake" the dependency tree and remove unused code from the final distributable. Because of this, relying on the official @atproto/api package will cause the entire lexicon namespaces used throughout Bluesky's codebase to be inherited, including all Ozone-specific and internal lexicon definitions. This hurts the bundle size of all apps, including the official Bluesky app.

The Custom Codegen Workaround

The "official" way of working around the bundle issues above is for projects to perform their own codegen, instead of relying on the official @atproto/api package. However, this workaround introduces its own significant pain points that make it difficult to maintain projects over time.

Manual lexicon curation: Developers must manually fetch, copy, and keep up-to-date the source lexicon schema definitions they're working with, so that the @atproto/lex-cli lexgen can build only the code that will actually be used during runtime. This manual process is error-prone and requires constant vigilance to stay synchronized with upstream changes.

Lost utilities: All the useful utilities for working with the app.bsky lexicons present in @atproto/api are no longer available when running custom codegen. This means those utilities must be rewritten and maintained separately, duplicating effort and creating maintenance burden.

Slow performance: The current codegen tool is quite slow. To give an idea, pnpm run codegen takes about 28 seconds for @atproto/api alone on my 2023 MacBook M3 Pro. Because it is so slow to build, the result of codegen is currently checked into the codebase. This causes pretty noisy diffs, even when no runtime code is affected, making pull requests harder to review and cluttering git history.

Unclear dependencies: Managing your own codegen means you need to install multiple dependencies to your project (@atproto/xrpc, @atproto/lexicon, multiformats), the purpose of which is not immediately clear to developers new to development in the Atmosphere. This creates additional friction during onboarding and setup.

Limited Architecture Patterns

Currently, the codegen tool operates in one of two modes: "client" or "server". However, this binary choice doesn't reflect the reality of many atproto applications. Some codebases need to function as both. For example, a backend service that makes Lexicon requests to other services while also serving its own endpoints. Other use cases don't fit cleanly into either category, such as a worker that consumes and validates records from the Firehose without acting as a traditional client or server. This inflexibility forces developers to choose between incomplete code generation or maintaining custom tooling to bridge the gap.

Looking ahead

The issues outlined above create significant friction for third-party developers building on the Atmosphere. While the current tooling has served the core atproto ecosystem well, these pain points make it challenging for teams to build efficient, production-ready applications using the Lexicon tooling provided by Bluesky.

That said, there may be some exciting developments on the horizon to help address these challenges, so stay tuned! In the meantime, we'd love to hear your thoughts: which of these issues impacts your development workflow the most? Are there other pain points we haven't covered?