Development

How the repository is organized, how to build and test it, and where to make common changes. For the contribution process, see CONTRIBUTING.md.

Setup#

Prerequisites: Git, a current stable Rust toolchain (rustup), and Node.js 20.19 or newer with npm. For the desktop app, also Tauri's system libraries (on Debian and Ubuntu, sudo apt-get install libwebkit2gtk-4.1-dev).

git clone https://github.com/sanskarIN/RepoDNA.git
cd RepoDNA
cargo xtask setup

cargo xtask setup checks the prerequisites and then does what you can also do step by step:

npm ci                              # web, schema, visualization, and desktop tooling
npm run build -w @repodna/web       # the web interface, embedded in the binary
cargo build -p repodna-cli          # the repodna binary
cargo xtask fixtures                # fixture repositories in fixtures/generated/

Run it:

cargo run -p repodna-cli -- analyze fixtures/generated/history
cargo run -p repodna-cli -- serve

Repository layout#

crates/
  repodna-core          domain model, configuration, evidence, findings, artifact I/O, schemas
  repodna-discovery     file walking, classification, safe reading, archive extraction
  repodna-parser        language registry and lexical analyzers
  repodna-git           hardened Git runner, history, refs, trees, URL safety
  repodna-dependencies  manifests and lockfiles per ecosystem
  repodna-architecture  modules, import resolution, graphs, cycles, layers
  repodna-quality       complexity, duplication, similarity, markers, dead code, hotspots
  repodna-security      secrets, risky patterns, permissions
  repodna-project       tests, build systems, commands, CI, documentation
  repodna-evolution     epochs, events, snapshots, story, module ages
  repodna-engine        the pipeline: input preparation, stages, insights, fingerprint
  repodna-store         SQLite index, stored artifacts, per-file cache
  repodna-report        Markdown, HTML, cards, badges, CSV, CI summaries, comparisons
  repodna-ai            optional AI explanations
  repodna-plugin        plugin discovery, manifests, and the analyzer protocol
  repodna-app           services shared by the CLI, the server, and the desktop app
  repodna-server        the local server behind `repodna serve`
  repodna-cli           the `repodna` binary
  repodna-wasm          the engine and reports as WebAssembly, for the web version
  repodna-testkit       deterministic Git repositories and fixtures for tests
apps/
  web                   React + TypeScript web interface (Vite)
  desktop               Tauri desktop app (its own Cargo workspace in src-tauri/)
packages/
  schema                TypeScript types generated from the artifact schema
  visualization         layout, color, and formatting helpers for charts
  wasm                  runs repodna-wasm in browsers and Node.js
packaging/              the container images, the npm packages, the GitHub Action, and the
                        documentation site
plugins/                example plugins
schemas/                published JSON Schemas
fixtures/               fixture documentation (repositories are generated)
benchmarks/             benchmark results and method
xtask/                  development tasks: fixtures and benchmarks
docs/                   this documentation

See architecture for how the pieces fit together.

Checks#

These are the checks CI runs. Run them before opening a pull request:

cargo fmt --all --check
cargo clippy --workspace --all-targets --locked -- -D warnings
cargo test --workspace --locked
RUSTDOCFLAGS="-D warnings" cargo doc --workspace --no-deps --locked

npm run format:check
npm run check-generated -w @repodna/schema
npm run typecheck
npm test
npm run build -w @repodna/web

The desktop app is checked separately (it needs the web build first):

cargo fmt --manifest-path apps/desktop/src-tauri/Cargo.toml --check
cargo clippy --manifest-path apps/desktop/src-tauri/Cargo.toml --all-targets --features custom-protocol --locked -- -D warnings
cargo test --manifest-path apps/desktop/src-tauri/Cargo.toml --features custom-protocol --locked

Dependencies are checked with cargo-deny: cargo deny check and cargo deny --manifest-path apps/desktop/src-tauri/Cargo.toml check.

THIRD-PARTY-NOTICES.txt lists the licenses of the Rust crates and npm packages that the downloads include, and ships with the command line, the desktop app, and the web interface. After adding, removing, or updating a dependency, regenerate it with cargo xtask notices (it needs npm ci first); CI fails with cargo xtask notices --check when it is out of date. A dependency under a license the task does not know makes it stop and say so.

npm run format formats the TypeScript code; cargo fmt --all formats the Rust code.

Tests#

• Unit tests live next to the code in each crate (#[cfg(test)] modules) and in *.test.ts(x) files for the web packages.
• Integration tests: crates/repodna-app/tests/fixtures.rs analyzes every fixture repository and checks what RepoDNA finds; crates/repodna-cli/tests/cli.rs runs the binary; crates/repodna-server/tests/server.rs exercises the API and its security checks.
• Web view tests render every view against the bundled demo artifact (apps/web/public/demo/repodna.json).
• repodna-testkit builds Git repositories with fixed authors and dates, so commit hashes and results are deterministic. Tests that need Git skip themselves when it is missing.

The web interface#

npm run dev -w @repodna/web          # http://localhost:5173

In development, API requests are proxied to a running repodna serve on port 7878. Start the server in another terminal and open http://localhost:5173/?token=TOKEN with the token it printed. Without a server, the interface still opens the demo and analysis files.

packages/schema/src/generated.ts is generated from schemas/repodna-artifact.schema.json; after changing the Rust model, regenerate both:

cargo run -p repodna-cli -- schema artifact > schemas/repodna-artifact.schema.json
cargo run -p repodna-cli -- schema config > schemas/repodna-config.schema.json
npm run generate -w @repodna/schema

A test fails when the published schemas are out of date.

The WebAssembly analysis#

The web version analyzes folders in the browser with crates/repodna-wasm built for wasm32-wasip1, which packages/wasm runs in a worker. Build it, and name it in REPODNA_ENGINE to include it in the web interface, for npm run dev as for a build:

rustup target add wasm32-wasip1
cargo build -p repodna-wasm --target wasm32-wasip1 --profile web-engine
REPODNA_ENGINE=target/wasm32-wasip1/web-engine/repodna-wasm.wasm npm run dev -w @repodna/web

npm test -w @repodna/wasm runs the program's tests when it has been built, and cargo clippy -p repodna-wasm --target wasm32-wasip1 checks the code that only builds for it. Without REPODNA_ENGINE, the interface works as before and offers no analysis in the browser; repodna serve and the desktop app never include it, since they analyze on the machine.

The documentation site#

The guides in docs/ are also published as a site at https://sanskarin.github.io/RepoDNA/docs/, beside the web version, with search, the changelog, the roadmap, and the project's policies. packaging/docs/build.mjs builds it:

node packaging/docs/build.mjs --out apps/web/dist/docs --check
python3 -m http.server --directory apps/web/dist 8000   # then open http://localhost:8000/docs/

Write the guides as for GitHub: relative links between Markdown files, and anchors as GitHub makes them. On the site, a link to another guide leads to its page and a link to any other file of the repository leads to it on GitHub. --check, the documentation site's tests in CI, and the Web version workflow fail on a link to a file or a heading that does not exist. The sidebar follows the sections and links of docs/index.md, so a new guide belongs there too; a page it does not link to is listed under More.

The desktop app#

npm run dev -w @repodna/desktop       # the app with live reloading
npm run bundle -w @repodna/desktop    # installers in apps/desktop/src-tauri/target/release/bundle/

See apps/desktop/README.md.

Common changes#

Adding or improving a language#

Built-in languages are data in crates/repodna-parser/src/builtin.rs: file names, comment and string syntax, import and symbol patterns, body style, and complexity keywords. Add a LanguageSpec there, then a test with a small sample in the same file or in symbols.rs/imports.rs. The test every_builtin_pattern_compiles_and_ids_are_unique compiles every pattern. If imports should resolve to files, extend crates/repodna-architecture/src/resolve.rs. Languages can also ship as plugins without changing RepoDNA.

Adding a dependency ecosystem#

Implement EcosystemProvider in crates/repodna-dependencies/src/providers/, register it in builtin_providers, and add tests with real-world manifest and lockfile samples. Providers only read file contents; they must never run a package manager or use the network.

Adding an analyzer or a finding#

1. Put the analysis in the crate it belongs to (or a new crate), producing types from repodna-core's model. Add fields to the model with #[serde(default)] so older artifacts stay readable.
2. Wire it into the pipeline in crates/repodna-engine/src/pipeline.rs. A new stage also needs an entry in Stage and the profiles in crates/repodna-core/src/config/profile.rs.
3. Findings use Finding::new(rule, subject, category, severity, confidence, title) with a summary, rationale, method, evidence, limitations, and next steps. Rule identifiers are area.kebab-case-name; document new rules in the rules reference.
4. Regenerate the schemas, show the result in a report section and a web view, and add a fixture or test that proves the signal is found and that its false positives are not.

Adding a metric#

Metrics are defined in crates/repodna-engine/src/fingerprint.rs with an identifier, label, value, unit, definition, method, and confidence. They appear in the metrics appendix, the CSV metrics table, and the web interface.

Adding a visualization#

Charts in the web interface are SVG components in apps/web/src/charts, built on the helpers in packages/visualization (layered graph layout, treemap, heatmap, scales, color palettes, number formatting). Every chart needs a table view and must not rely on color alone. Charts in HTML reports are rendered in Rust in crates/repodna-report/src/charts.rs.

Adding a report section#

Sections are listed in crates/repodna-report/src/sections.rs, and their content is built in crates/repodna-report/src/content/ as format-independent blocks, which the Markdown, HTML, and terminal renderers turn into output.

Adding an AI provider#

Implement the AiProvider trait in crates/repodna-ai/src/provider/, add the provider kind to AiProviderKind in crates/repodna-core/src/config/mod.rs, and build it in build_provider. Report remote() honestly: remote providers need the user's consent.

Database migrations#

The storage schema is a list of migrations in crates/repodna-store/src/schema.rs; the schema version is the number of migrations. Append a migration, never edit an existing one. Migrations run in a transaction, so a failed upgrade leaves the previous data intact.

Fixtures and benchmarks#

cargo xtask fixtures                    # write every fixture to fixtures/generated/
cargo build --release -p repodna-cli
cargo xtask bench --runs 5              # the table in benchmarks/README.md

Releasing#

1. Update the version everywhere it appears: Cargo.toml ([workspace.package] and the internal dependencies), apps/desktop/src-tauri/Cargo.toml, apps/desktop/src-tauri/tauri.conf.json, and the package.json files (root, apps/web, apps/desktop, packages/schema, packages/visualization, packages/wasm). Run cargo build and npm install to update the lockfiles.
2. Add a section for the version to CHANGELOG.md, and point the installation steps and examples at it: README.md, docs/installation.md, docs/plugins.md, docs/repository-dna.md, and examples/ci/repodna.yml. For a new minor or major version, also name it in the supported versions of SECURITY.md.
3. Regenerate the schemas and the bundled demo analysis if the model changed, and the third-party notices (cargo xtask notices) if dependencies changed.
4. Add the version's images to the images branch: its screenshots in screenshots/v1.3.2, with the same views as the earlier versions, and a column and a section for it in screenshots/README.md; and its promo images and Project DNA cards in media/v1.3.2, shown as the newest in media/README.md. Point the screenshots in the README and the guides at the new folder. On the releases-info branch, add v1.3.2/release-notes.md with the notes prepared for the release, v1.3.2/pull-requests.md, and a row in its README.md.
5. Commit, then create and push an annotated tag: git tag -a v1.3.2 -m "RepoDNA 1.3.2" and git push origin v1.3.2. The release notes show the screenshots of the version from the images branch.
6. Once the release is published, on the releases-info branch, replace v1.3.2/release-notes.md with the notes as published, and add v1.3.2/README.md with its notes, its downloads and their sizes, and how to install it.

The images and releases-info branches share no history with main, so that the code does not carry the images and release pages. Work on one in a folder of its own, for example with git fetch origin images and git worktree add ../RepoDNA-images images. Never rename or remove a version's folder on them: the README, the guides, and the release notes link to them.

The release workflow checks that the tag matches the workspace version and the changelog, builds the command line for Linux, macOS, and Windows (x64 and Arm), the WebAssembly analysis, and the desktop installers, and publishes them with checksums. It also publishes these packages to GitHub Packages, and the npm packages to npmjs.com too:

The GitHub Action is action.yml at the root and packaging/action/install.sh; a tag makes it usable as sanskarIN/RepoDNA@<tag>, and it installs the release of that tag.

GitHub makes a new package private. After the first release that publishes it, make each package public once in its settings on GitHub (Package settings > Change visibility). To look at the npm packages before a release, stage them with node packaging/npm/build.mjs (add --binaries DIR for the command line packages, and --wasm FILE for the WebAssembly package) and test the scripts with node --test "packaging/npm/test/*.test.mjs".

Publishing to npmjs.com, where the packages install without a token, needs the repository secret NPM_TOKEN (Settings > Secrets and variables > Actions): a granular access token of the npm account that owns the @sanskarin scope, with read and write access to packages and allowed to publish without two-factor authentication. npm limits how long such a token lasts, so renew it before a release when it is about to expire. npm is also retiring such tokens: from January 2027, a publish made with one waits under Staged Packages on npmjs.com until you approve it with two-factor authentication. Trusted publishing replaces the token: once the packages are on npmjs.com, add this repository and the workflow file as a trusted publisher in each package's settings there, and set the repository variable NPM_TRUSTED_PUBLISHING to true. Without either, releases skip npmjs.com and say so.

To publish the npm packages of a release that is already out, for example after adding the token, run the npm packages workflow from the Actions tab with the release's tag. It packages the archives attached to the release and publishes what GitHub Packages and npmjs.com do not have yet. A release older than the newest one on a registry goes under the previous tag there, so npm install keeps picking the newest release.

To try a release before tagging it, run the Release workflow by hand from the Actions tab with an empty tag: it builds every file from the selected branch and keeps them as downloads of the run, without publishing anything.