michal/tit

Browse tree · Show commit · Download archive

Diff

519080ccb711 → 033de95f5eff

@@ -105,6 +105,53 @@
 It does not activate the restored instance. To activate it, stop the old server
 and explicitly start `tit --config /srv/tit-restored/config.toml serve`.
 
+## Diagnostics and repair
+
+Run the read-only instance checks with:
+
+```text
+tit --config /srv/tit/config.toml doctor
+```
+
+Also check one or more backup archives with:
+
+```text
+tit --config /srv/tit/config.toml doctor --backup /var/backups/tit-2026-07-23.tar
+```
+
+Doctor checks configuration, private permissions, the schema, record
+relations, indexes, incomplete intents, Git refs and reachable objects,
+quarantine debris, the SSH host key, and each supplied backup. It does not
+change the instance.
+
+Use the separate repair commands only after you review the doctor error:
+
+```text
+tit --config /srv/tit/config.toml repair intents
+tit --config /srv/tit/config.toml repair quarantine
+```
+
+Stop the server before repair. Intent repair uses the normal recovery rules.
+Quarantine repair refuses to run while an incomplete intent exists.
+
+Inspect one typed record as JSON:
+
+```text
+tit --config /srv/tit/config.toml inspect account alice
+tit --config /srv/tit/config.toml inspect repository alice example
+tit --config /srv/tit/config.toml inspect intent 0123456789abcdef0123456789abcdef
+```
+
+Write all SQLite rows as deterministic JSON Lines:
+
+```text
+tit --config /srv/tit/config.toml dump >tit-dump.jsonl
+```
+
+The dump can contain credential hashes, session hashes, token hashes, and SSH
+public keys. Store it as a secret. The dump is for inspection and comparison;
+it is not a restore format.
+
 An authenticated account can create a repository with SSH:
 
 ```text
@@ -540,3 +587,18 @@
 database validation, and restored Git object access. Read the
 [backup and restore architectural decision record](docs/adr/0029-backup-and-restore.md)
 for the archive, gate, credential, and activation contracts.
+
+## Milestone 6.3 gate
+
+Install stock Git and stock `ssh-keygen`. Then, run the diagnostics gate:
+
+```text
+./scripts/check-m6-3
+```
+
+This command tests read-only doctor checks, explicit repair, typed inspection,
+deterministic JSON Lines output, damaged Git state, incomplete intents,
+quarantine debris, missing indexes, unsafe permissions, and changed backup
+data. Read the
+[read-only diagnostics architectural decision record](docs/adr/0030-read-only-diagnostics.md)
+for the check, repair, inspection, and dump contracts.

@@ -1,0 +1,95 @@
+# Architectural decision record 0030: read-only diagnostics
+
+Status: Accepted
+
+Date: 2026-07-23
+
+## Context
+
+The initial `tit doctor` command checked only the SQLite schema version,
+database pages, and foreign keys. It opened a read-write connection and applied
+the normal connection configuration. It did not check the filesystem, Git
+state, incomplete cross-store operations, or backup archives.
+
+An operator also needed stable ways to examine records. Direct SQLite queries
+require knowledge of the schema and can accidentally change the database.
+
+## Decision
+
+`tit doctor` opens SQLite with `SQLITE_OPEN_READ_ONLY`. It does not take the
+instance lock, migrate a schema, recover an intent, remove quarantine data, or
+create a file. It checks:
+
+- the instance, configuration, database, repository-root, and SSH host-key
+  types and permissions;
+- the current schema version, SQLite integrity, foreign keys, and all required
+  indexes;
+- incomplete Git operation intents and pull-request ref intents;
+- the database record for each repository and each repository directory;
+- the Git object format, refs, and all reachable Git objects;
+- unknown repository-root entries and quarantine debris; and
+- each backup archive supplied with `--backup`.
+
+The configuration parser runs before these checks, so configuration syntax and
+values are part of the doctor result. A missing SSH host key is an error because
+the instance is not ready to preserve its SSH identity.
+
+Add separate `repair intents` and `repair quarantine` commands. Each repair
+command takes the instance lock. Intent repair uses the normal recovery logic.
+Quarantine repair refuses to run while an incomplete intent exists. Doctor
+never calls either repair command.
+
+Add typed `inspect account`, `inspect repository`, and `inspect intent`
+commands. Each command returns one version-independent JSON object for the
+selected record type. Account inspection does not return the recovery hash or
+canonical public-key text.
+
+Add `tit dump`. It streams one JSON object for each SQLite row in deterministic
+table and primary-key order. Each object identifies the table and gives each
+column name, SQLite value type, and value. BLOB values and invalid UTF-8 text
+use lowercase hexadecimal. Valid UTF-8 text stays text. Real values use a
+deterministic 17-digit exponential form.
+
+The dump is a raw operational artifact. It can contain credential hashes,
+session hashes, feed-token hashes, invitation hashes, and SSH public keys.
+Operators must store it as a secret.
+
+## Failure and threat cases
+
+Opening a normal `Store` can migrate an old schema and can create a migration
+backup. Diagnostics use a separate read-only constructor, so a doctor,
+inspection, or dump cannot do these operations.
+
+Incomplete intents and quarantine directories can be necessary for recovery.
+Doctor reports them and leaves them unchanged. The explicit repair commands
+serialize with the server through the instance lock.
+
+A repository directory can be a symbolic link or can have an identifier that
+has no database record. Doctor rejects both cases. It also opens the repository
+without a runtime Git command and walks reachable objects with the established
+object-count limit.
+
+A damaged backup can change between validation passes. Backup validation checks
+the manifest entry type, size, and checksum. Restore checks them again while it
+extracts each file.
+
+## Evidence
+
+The CLI tests run doctor on a correct instance and on instances with a foreign
+key violation, a missing index, an incomplete intent, quarantine debris, an
+invalid Git ref, unsafe permissions, and a changed backup. The test confirms
+that doctor leaves the incomplete intent unchanged. It then runs each explicit
+repair command and checks the result.
+
+The same test checks typed account, repository, and intent JSON. It runs the
+JSON Lines dump two times and checks byte-for-byte equality and valid typed
+rows.
+
+## Consequences
+
+Doctor can run while the server is active because it does not change state.
+The result is a point-in-time sequence of checks, not a global snapshot. Use an
+online backup when one coherent cross-store snapshot is necessary.
+
+The JSON Lines dump exposes storage details intentionally. It is suitable for
+external inspection and comparison, but it is not a restore format.

@@ -1,0 +1,5 @@
+#!/bin/sh
+set -eu
+
+./scripts/check
+cargo test --locked --release --test cli doctor_

@@ -86,6 +86,13 @@
     Ok(())
 }
 
+pub(crate) fn check_archive(archive_path: &Path) -> Result<(), BackupError> {
+    validate_absolute_clean(archive_path)?;
+    let manifest = read_manifest(archive_path)?;
+    let expected = expected_files(&manifest)?;
+    verify_archive(archive_path, &expected)
+}
+
 fn create_archive(
     instance_dir: &Path,
     config_path: &Path,

@@ -50,8 +50,24 @@
     Serve,
     /// Create a single-use signup invitation
     InviteCode,
-    /// Check the instance database
-    Doctor,
+    /// Check the instance without changing it
+    Doctor {
+        /// Also check the manifest and checksums in FILE
+        #[arg(long = "backup", value_name = "FILE")]
+        backups: Vec<PathBuf>,
+    },
+    /// Show one typed instance record as JSON
+    Inspect {
+        #[command(subcommand)]
+        command: InspectCommand,
+    },
+    /// Write all SQLite rows as deterministic JSON Lines
+    Dump,
+    /// Run an explicit repair operation
+    Repair {
+        #[command(subcommand)]
+        command: RepairCommand,
+    },
     /// Create a backup archive
     Backup {
         /// Write the backup archive to FILE
@@ -74,6 +90,24 @@
         #[command(subcommand)]
         command: AdminCommand,
     },
+}
+
+#[derive(Clone, Debug, Subcommand)]
+pub(crate) enum RepairCommand {
+    /// Recover incomplete Git and pull-request ref intents
+    Intents,
+    /// Remove quarantine debris after all intents are complete
+    Quarantine,
+}
+
+#[derive(Clone, Debug, Subcommand)]
+pub(crate) enum InspectCommand {
+    /// Show an account and its SSH key metadata
+    Account { username: String },
+    /// Show a repository record after Git validation
+    Repository { owner: String, slug: String },
+    /// Show a Git operation intent
+    Intent { id: String },
 }
 
 #[derive(Clone, Debug, Subcommand)]

@@ -1,0 +1,263 @@
+use std::collections::HashSet;
+use std::fs;
+use std::os::unix::fs::PermissionsExt;
+use std::path::{Path, PathBuf};
+
+use thiserror::Error;
+
+use crate::backup::{self, BackupError};
+use crate::config::Config;
+use crate::git::repository::{GitRepository, GitRepositoryError};
+use crate::instance::REPOSITORY_DIRECTORY;
+use crate::serve::{self, ServeError};
+use crate::store::{
+    AccountInspection, DATABASE_FILE, DumpRow, GitIntentInspection, RepositoryInspection, Store,
+    StoreError,
+};
+
+const REQUIRED_INDEXES: &[&str] = &[
+    "audit_event_correlation",
+    "audit_event_history",
+    "feed_token_account_active",
+    "feed_token_repository_active",
+    "git_operation_intent_incomplete",
+    "issue_assignee_account",
+    "issue_comment_history",
+    "issue_label_label",
+    "issue_repository_state",
+    "label_repository_name",
+    "login_nonce_active",
+    "m1a_child_state_parent",
+    "m1a_parent_created_at",
+    "pull_request_ref_intent_incomplete",
+    "pull_request_repository_state",
+    "pull_request_review_status",
+    "pull_request_review_timeline",
+    "pull_request_revision_history",
+    "repository_collaborator_account",
+    "repository_event_feed",
+    "repository_event_issue_timeline",
+    "repository_event_pull_request_timeline",
+    "signup_invitation_active",
+    "ssh_public_key_account",
+    "watch_account_activity",
+    "web_session_account_active",
+];
+
+pub(crate) fn doctor(config: &Config, backups: &[PathBuf]) -> Result<(), DiagnosticError> {
+    check_private_directory(&config.instance_dir)?;
+    check_private_file(&config.config_path)?;
+    let database = config.instance_dir.join(DATABASE_FILE);
+    check_private_file(&database)?;
+    crate::store::doctor(&config.instance_dir)?;
+
+    let store = Store::open_read_only(&database)?;
+    let indexes: HashSet<_> = store.index_names()?.into_iter().collect();
+    for required in REQUIRED_INDEXES {
+        if !indexes.contains(*required) {
+            return Err(DiagnosticError::MissingIndex((*required).to_owned()));
+        }
+    }
+    let git_intents = store.incomplete_git_intents()?;
+    let pull_request_intents = store.incomplete_pull_request_ref_intents()?;
+    if !git_intents.is_empty() || !pull_request_intents.is_empty() {
+        return Err(DiagnosticError::IncompleteIntents {
+            git: git_intents.len(),
+            pull_request: pull_request_intents.len(),
+        });
+    }
+    check_repositories(&config.instance_dir, &store)?;
+    serve::check_host_key(&config.instance_dir)?;
+    for archive in backups {
+        backup::check_archive(archive)?;
+    }
+    Ok(())
+}
+
+pub(crate) fn inspect_account(
+    config: &Config,
+    username: &str,
+) -> Result<AccountInspection, DiagnosticError> {
+    let store = Store::open_read_only(&config.instance_dir.join(DATABASE_FILE))?;
+    Ok(store.inspect_account(username)?)
+}
+
+pub(crate) fn inspect_repository(
+    config: &Config,
+    owner: &str,
+    slug: &str,
+) -> Result<RepositoryInspection, DiagnosticError> {
+    let store = Store::open_read_only(&config.instance_dir.join(DATABASE_FILE))?;
+    let repository = store.repository(owner, slug)?;
+    check_repository(&config.instance_dir, &repository)?;
+    Ok(repository.into())
+}
+
+pub(crate) fn inspect_intent(
+    config: &Config,
+    id: &str,
+) -> Result<GitIntentInspection, DiagnosticError> {
+    let store = Store::open_read_only(&config.instance_dir.join(DATABASE_FILE))?;
+    Ok(store.inspect_git_intent(id)?)
+}
+
+pub(crate) fn dump(
+    config: &Config,
+    visit: impl FnMut(DumpRow) -> bool,
+) -> Result<(), DiagnosticError> {
+    let store = Store::open_read_only(&config.instance_dir.join(DATABASE_FILE))?;
+    Ok(store.dump_rows(visit)?)
+}
+
+fn check_repositories(instance_dir: &Path, store: &Store) -> Result<(), DiagnosticError> {
+    let root = instance_dir.join(REPOSITORY_DIRECTORY);
+    check_private_directory(&root)?;
+    let repositories = store.all_repositories()?;
+    let expected: HashSet<_> = repositories
+        .iter()
+        .map(|repository| format!("{}.git", repository.id))
+        .collect();
+    for repository in &repositories {
+        check_repository(instance_dir, repository)?;
+    }
+    for entry in fs::read_dir(&root).map_err(|source| DiagnosticError::Io {
+        path: root.clone(),
+        source,
+    })? {
+        let entry = entry.map_err(|source| DiagnosticError::Io {
+            path: root.clone(),
+            source,
+        })?;
+        let Some(name) = entry.file_name().to_str().map(str::to_owned) else {
+            return Err(DiagnosticError::UnknownRepositoryEntry(entry.path()));
+        };
+        if !expected.contains(&name) {
+            return Err(DiagnosticError::UnknownRepositoryEntry(entry.path()));
+        }
+    }
+    Ok(())
+}
+
+fn check_repository(
+    instance_dir: &Path,
+    repository: &crate::store::RepositoryRecord,
+) -> Result<(), DiagnosticError> {
+    let path = instance_dir
+        .join(REPOSITORY_DIRECTORY)
+        .join(format!("{}.git", repository.id));
+    let metadata = fs::symlink_metadata(&path).map_err(|source| DiagnosticError::Io {
+        path: path.clone(),
+        source,
+    })?;
+    if metadata.file_type().is_symlink() || !metadata.file_type().is_dir() {
+        return Err(DiagnosticError::UnsafePath(path));
+    }
+    let git = GitRepository::open(&path)?;
+    let format = match git.object_format() {
+        gix::hash::Kind::Sha1 => "sha1",
+        gix::hash::Kind::Sha256 => "sha256",
+        _ => return Err(DiagnosticError::UnsupportedObjectFormat),
+    };
+    if format != repository.object_format {
+        return Err(DiagnosticError::ObjectFormat(repository.id.clone()));
+    }
+    git.integrity_check()?;
+
+    let quarantine = path.join("objects").join("tit-quarantine");
+    match fs::symlink_metadata(&quarantine) {
+        Ok(metadata) => {
+            if metadata.file_type().is_symlink() || !metadata.file_type().is_dir() {
+                return Err(DiagnosticError::UnsafePath(quarantine));
+            }
+            if fs::read_dir(&quarantine)
+                .map_err(|source| DiagnosticError::Io {
+                    path: quarantine.clone(),
+                    source,
+                })?
+                .next()
+                .is_some()
+            {
+                return Err(DiagnosticError::QuarantineDebris(quarantine));
+            }
+        }
+        Err(error) if error.kind() == std::io::ErrorKind::NotFound => {}
+        Err(source) => {
+            return Err(DiagnosticError::Io {
+                path: quarantine,
+                source,
+            });
+        }
+    }
+    Ok(())
+}
+
+fn check_private_directory(path: &Path) -> Result<(), DiagnosticError> {
+    let metadata = fs::symlink_metadata(path).map_err(|source| DiagnosticError::Io {
+        path: path.to_owned(),
+        source,
+    })?;
+    if metadata.file_type().is_symlink() || !metadata.file_type().is_dir() {
+        return Err(DiagnosticError::UnsafePath(path.to_owned()));
+    }
+    let mode = metadata.permissions().mode() & 0o777;
+    if mode & 0o077 != 0 {
+        return Err(DiagnosticError::Permissions {
+            path: path.to_owned(),
+            mode,
+        });
+    }
+    Ok(())
+}
+
+fn check_private_file(path: &Path) -> Result<(), DiagnosticError> {
+    let metadata = fs::symlink_metadata(path).map_err(|source| DiagnosticError::Io {
+        path: path.to_owned(),
+        source,
+    })?;
+    if metadata.file_type().is_symlink() || !metadata.file_type().is_file() {
+        return Err(DiagnosticError::UnsafePath(path.to_owned()));
+    }
+    let mode = metadata.permissions().mode() & 0o777;
+    if mode & 0o077 != 0 {
+        return Err(DiagnosticError::Permissions {
+            path: path.to_owned(),
+            mode,
+        });
+    }
+    Ok(())
+}
+
+#[derive(Debug, Error)]
+pub(crate) enum DiagnosticError {
+    #[error("diagnostic path is unsafe: {0}")]
+    UnsafePath(PathBuf),
+    #[error("diagnostic path permissions for {path} are {mode:o}, expected owner-only access")]
+    Permissions { path: PathBuf, mode: u32 },
+    #[error("required database index does not exist: {0}")]
+    MissingIndex(String),
+    #[error(
+        "incomplete intents exist: {git} Git operation intents and {pull_request} pull-request ref intents"
+    )]
+    IncompleteIntents { git: usize, pull_request: usize },
+    #[error("repository directory has an unknown entry: {0}")]
+    UnknownRepositoryEntry(PathBuf),
+    #[error("repository has quarantine debris: {0}")]
+    QuarantineDebris(PathBuf),
+    #[error("repository object format does not match its database record: {0}")]
+    ObjectFormat(String),
+    #[error("repository object format is not supported")]
+    UnsupportedObjectFormat,
+    #[error("diagnostic I/O failed for {path}: {source}")]
+    Io {
+        path: PathBuf,
+        source: std::io::Error,
+    },
+    #[error(transparent)]
+    Store(#[from] StoreError),
+    #[error(transparent)]
+    Git(#[from] GitRepositoryError),
+    #[error(transparent)]
+    HostKey(#[from] ServeError),
+    #[error(transparent)]
+    Backup(#[from] BackupError),
+}

@@ -10,6 +10,7 @@
 mod cli;
 mod config;
 mod control;
+mod diagnostics;
 mod domain;
 mod feed;
 mod feed_token;
@@ -23,6 +24,7 @@
 mod markdown;
 mod policy;
 mod pull_request;
+mod repair;
 mod repository;
 mod search;
 mod serve;
@@ -38,8 +40,8 @@
 use clap::Parser;
 
 use crate::cli::{
-    AccountCommand, AdminCommand, Cli, CollaboratorRole, Command, ObjectFormat, RepositoryCommand,
-    RepositoryVisibility, SetupCommand,
+    AccountCommand, AdminCommand, Cli, CollaboratorRole, Command, InspectCommand, ObjectFormat,
+    RepairCommand, RepositoryCommand, RepositoryVisibility, SetupCommand,
 };
 
 #[tokio::main]
@@ -82,13 +84,28 @@
                     }
                 }
             }
-            Some(Command::Doctor) => match store::doctor(&config.instance_dir) {
+            Some(Command::Doctor { backups }) => match diagnostics::doctor(&config, &backups) {
                 Ok(()) => ExitCode::SUCCESS,
                 Err(error) => {
                     eprintln!("tit: {error}");
                     ExitCode::FAILURE
                 }
             },
+            Some(Command::Inspect { command }) => run_inspect_command(&config, command),
+            Some(Command::Dump) => run_dump_command(&config),
+            Some(Command::Repair { command }) => {
+                let result = match command {
+                    RepairCommand::Intents => repair::intents(&config.instance_dir),
+                    RepairCommand::Quarantine => repair::quarantine(&config.instance_dir),
+                };
+                match result {
+                    Ok(()) => ExitCode::SUCCESS,
+                    Err(error) => {
+                        eprintln!("tit: {error}");
+                        ExitCode::FAILURE
+                    }
+                }
+            }
             Some(Command::Backup { output }) => run_backup(&config, &output).await,
             Some(Command::Restore { .. }) => {
                 unreachable!("the restore command runs before configuration is loaded")
@@ -129,6 +146,71 @@
                 command: AdminCommand::Audit { limit },
             }) => run_audit_command(&config.instance_dir, limit),
         },
+        Err(error) => {
+            eprintln!("tit: {error}");
+            ExitCode::FAILURE
+        }
+    }
+}
+
+fn run_inspect_command(config: &config::Config, command: InspectCommand) -> ExitCode {
+    let result = match command {
+        InspectCommand::Account { username } => {
+            serialize_inspection(diagnostics::inspect_account(config, &username))
+        }
+        InspectCommand::Repository { owner, slug } => {
+            serialize_inspection(diagnostics::inspect_repository(config, &owner, &slug))
+        }
+        InspectCommand::Intent { id } => {
+            serialize_inspection(diagnostics::inspect_intent(config, &id))
+        }
+    };
+    match result {
+        Ok(line) => match writeln!(io::stdout().lock(), "{line}") {
+            Ok(()) => ExitCode::SUCCESS,
+            Err(error) => {
+                eprintln!("tit: cannot write inspect information: {error}");
+                ExitCode::FAILURE
+            }
+        },
+        Err(error) => {
+            eprintln!("tit: {error}");
+            ExitCode::FAILURE
+        }
+    }
+}
+
+fn serialize_inspection(
+    result: Result<impl serde::Serialize, diagnostics::DiagnosticError>,
+) -> Result<String, Box<dyn std::error::Error>> {
+    Ok(serde_json::to_string(&result?)?)
+}
+
+fn run_dump_command(config: &config::Config) -> ExitCode {
+    let result = (|| -> Result<(), Box<dyn std::error::Error>> {
+        let mut output = io::stdout().lock();
+        let mut output_error = None;
+        diagnostics::dump(config, |row| {
+            let result = serde_json::to_writer(&mut output, &row)
+                .map_err(|error| Box::new(error) as Box<dyn std::error::Error>)
+                .and_then(|()| {
+                    writeln!(output).map_err(|error| Box::new(error) as Box<dyn std::error::Error>)
+                });
+            match result {
+                Ok(()) => true,
+                Err(error) => {
+                    output_error = Some(error);
+                    false
+                }
+            }
+        })?;
+        if let Some(error) = output_error {
+            return Err(error);
+        }
+        Ok(())
+    })();
+    match result {
+        Ok(()) => ExitCode::SUCCESS,
         Err(error) => {
             eprintln!("tit: {error}");
             ExitCode::FAILURE

@@ -1,0 +1,69 @@
+use std::fs;
+use std::path::{Path, PathBuf};
+
+use thiserror::Error;
+
+use crate::instance::{InstanceError, InstanceLock, REPOSITORY_DIRECTORY};
+use crate::pull_request::{PullRequestError, PullRequestService};
+use crate::store::{DATABASE_FILE, Store, StoreError};
+
+pub(crate) fn intents(instance_dir: &Path) -> Result<(), RepairError> {
+    let _lock = InstanceLock::acquire(instance_dir)?;
+    crate::store::doctor(instance_dir)?;
+    let database = instance_dir.join(DATABASE_FILE);
+    let repositories = instance_dir.join(REPOSITORY_DIRECTORY);
+    PullRequestService::new(&database, &repositories).recover()?;
+    Ok(())
+}
+
+pub(crate) fn quarantine(instance_dir: &Path) -> Result<(), RepairError> {
+    let _lock = InstanceLock::acquire(instance_dir)?;
+    crate::store::doctor(instance_dir)?;
+    let database = instance_dir.join(DATABASE_FILE);
+    let store = Store::open_read_only(&database)?;
+    if !store.incomplete_git_intents()?.is_empty()
+        || !store.incomplete_pull_request_ref_intents()?.is_empty()
+    {
+        return Err(RepairError::IncompleteIntents);
+    }
+    let repositories = store.all_repositories()?;
+    drop(store);
+    for repository in repositories {
+        let path = instance_dir
+            .join(REPOSITORY_DIRECTORY)
+            .join(format!("{}.git", repository.id))
+            .join("objects")
+            .join("tit-quarantine");
+        match fs::symlink_metadata(&path) {
+            Ok(metadata) if metadata.file_type().is_dir() && !metadata.file_type().is_symlink() => {
+                fs::remove_dir_all(&path).map_err(|source| RepairError::Io {
+                    path: path.clone(),
+                    source,
+                })?;
+            }
+            Ok(_) => return Err(RepairError::UnsafePath(path)),
+            Err(error) if error.kind() == std::io::ErrorKind::NotFound => {}
+            Err(source) => return Err(RepairError::Io { path, source }),
+        }
+    }
+    Ok(())
+}
+
+#[derive(Debug, Error)]
+pub(crate) enum RepairError {
+    #[error("repair path is unsafe: {0}")]
+    UnsafePath(PathBuf),
+    #[error("repair of quarantine debris requires all intents to be complete")]
+    IncompleteIntents,
+    #[error("repair I/O failed for {path}: {source}")]
+    Io {
+        path: PathBuf,
+        source: std::io::Error,
+    },
+    #[error(transparent)]
+    Instance(#[from] InstanceError),
+    #[error(transparent)]
+    Store(#[from] StoreError),
+    #[error(transparent)]
+    PullRequest(#[from] PullRequestError),
+}

@@ -145,6 +145,19 @@
     read_host_key(&path)
 }
 
+pub(crate) fn check_host_key(instance_dir: &Path) -> Result<(), ServeError> {
+    let path = instance_dir.join(HOST_KEY_FILE);
+    let metadata = fs::symlink_metadata(&path).map_err(|source| ServeError::HostKeyIo {
+        path: path.clone(),
+        source,
+    })?;
+    if metadata.file_type().is_symlink() || !metadata.file_type().is_file() {
+        return Err(ServeError::InvalidHostKeyFile(path));
+    }
+    read_host_key(&path)?;
+    Ok(())
+}
+
 fn create_host_key(path: &Path) -> Result<PrivateKey, ServeError> {
     let key = PrivateKey::random(&mut rng(), Algorithm::Ed25519)?;
     let encoded = key.to_openssh(LineEnding::LF)?;

@@ -4,7 +4,9 @@
 
 use rusqlite::OpenFlags;
 use rusqlite::backup::Backup;
+use rusqlite::types::ValueRef;
 use rusqlite::{Connection, OptionalExtension, TransactionBehavior};
+use serde::Serialize;
 use thiserror::Error;
 
 mod event;
@@ -226,6 +228,16 @@
     pub(crate) fn open_unmigrated(path: &Path) -> Result<Self, StoreError> {
         let connection = Connection::open(path)?;
         configure(&connection)?;
+        Ok(Self { connection })
+    }
+
+    pub(crate) fn open_read_only(path: &Path) -> Result<Self, StoreError> {
+        let connection = Connection::open_with_flags(
+            path,
+            OpenFlags::SQLITE_OPEN_READ_ONLY | OpenFlags::SQLITE_OPEN_NO_MUTEX,
+        )?;
+        connection.busy_timeout(BUSY_TIMEOUT)?;
+        connection.pragma_update(None, "foreign_keys", true)?;
         Ok(Self { connection })
     }
 
@@ -2933,6 +2945,158 @@
             .map_err(Into::into)
     }
 
+    pub(crate) fn index_names(&self) -> Result<Vec<String>, StoreError> {
+        let mut statement = self.connection.prepare(
+            "SELECT name FROM sqlite_schema
+             WHERE type = 'index' AND name NOT LIKE 'sqlite_%'
+             ORDER BY name",
+        )?;
+        statement
+            .query_map([], |row| row.get(0))?
+            .collect::<Result<Vec<_>, _>>()
+            .map_err(Into::into)
+    }
+
+    pub(crate) fn inspect_account(&self, username: &str) -> Result<AccountInspection, StoreError> {
+        let account = self
+            .connection
+            .query_row(
+                "SELECT id, username, is_administrator, state, created_at
+                 FROM account WHERE username = ?1",
+                [username],
+                |row| {
+                    Ok((
+                        row.get::<_, i64>(0)?,
+                        row.get::<_, String>(1)?,
+                        row.get::<_, bool>(2)?,
+                        row.get::<_, String>(3)?,
+                        row.get::<_, i64>(4)?,
+                    ))
+                },
+            )
+            .optional()?
+            .ok_or_else(|| StoreError::AccountNotFound(username.to_owned()))?;
+        let mut statement = self.connection.prepare(
+            "SELECT label, fingerprint, created_at, last_used_at, revoked_at
+             FROM ssh_public_key WHERE account_id = ?1 ORDER BY id",
+        )?;
+        let keys = statement
+            .query_map([account.0], |row| {
+                Ok(KeyInspection {
+                    label: row.get(0)?,
+                    fingerprint: row.get(1)?,
+                    created_at: row.get(2)?,
+                    last_used_at: row.get(3)?,
+                    revoked_at: row.get(4)?,
+                })
+            })?
+            .collect::<Result<Vec<_>, _>>()?;
+        Ok(AccountInspection {
+            record_type: "account",
+            username: account.1,
+            administrator: account.2,
+            state: account.3,
+            created_at: account.4,
+            keys,
+        })
+    }
+
+    pub(crate) fn inspect_git_intent(&self, id: &str) -> Result<GitIntentInspection, StoreError> {
+        self.connection
+            .query_row(
+                "SELECT id, repository_path, actor, quarantine_path, state, pack_name, created_at
+                 FROM git_operation_intent WHERE id = ?1",
+                [id],
+                |row| {
+                    Ok(GitIntentInspection {
+                        record_type: "git-intent",
+                        id: row.get(0)?,
+                        repository_path: row.get(1)?,
+                        actor: row.get(2)?,
+                        quarantine_path: row.get(3)?,
+                        state: row.get(4)?,
+                        pack_name: row.get(5)?,
+                        created_at: row.get(6)?,
+                    })
+                },
+            )
+            .optional()?
+            .ok_or_else(|| StoreError::Integrity(format!("Git intent does not exist: {id}")))
+    }
+
+    pub(crate) fn dump_rows(
+        &self,
+        mut visit: impl FnMut(DumpRow) -> bool,
+    ) -> Result<(), StoreError> {
+        let mut tables_statement = self.connection.prepare(
+            "SELECT name FROM sqlite_schema
+             WHERE type = 'table' AND name NOT LIKE 'sqlite_%'
+             ORDER BY name",
+        )?;
+        let tables = tables_statement
+            .query_map([], |row| row.get::<_, String>(0))?
+            .collect::<Result<Vec<_>, _>>()?;
+        for table in tables {
+            let columns = self.table_columns(&table)?;
+            let order = columns
+                .iter()
+                .filter(|column| column.primary_key > 0)
+                .map(|column| (column.primary_key, quote_identifier(&column.name)))
+                .collect::<Vec<_>>();
+            let order = if order.is_empty() {
+                "rowid".to_owned()
+            } else {
+                let mut order = order;
+                order.sort_by_key(|item| item.0);
+                order
+                    .into_iter()
+                    .map(|item| item.1)
+                    .collect::<Vec<_>>()
+                    .join(", ")
+            };
+            let sql = format!(
+                "SELECT * FROM {} ORDER BY {order}",
+                quote_identifier(&table)
+            );
+            let mut statement = self.connection.prepare(&sql)?;
+            let mut rows = statement.query([])?;
+            while let Some(row) = rows.next()? {
+                let mut values = Vec::with_capacity(columns.len());
+                for (index, column) in columns.iter().enumerate() {
+                    values.push(DumpColumn {
+                        name: column.name.clone(),
+                        value: dump_value(row.get_ref(index)?),
+                    });
+                }
+                let output = DumpRow {
+                    record_type: "sqlite-row",
+                    table: table.clone(),
+                    columns: values,
+                };
+                if !visit(output) {
+                    return Ok(());
+                }
+            }
+        }
+        Ok(())
+    }
+
+    fn table_columns(&self, table: &str) -> Result<Vec<TableColumn>, StoreError> {
+        let mut statement = self.connection.prepare(
+            "SELECT name, pk FROM pragma_table_info(?1)
+             ORDER BY cid",
+        )?;
+        statement
+            .query_map([table], |row| {
+                Ok(TableColumn {
+                    name: row.get(0)?,
+                    primary_key: row.get(1)?,
+                })
+            })?
+            .collect::<Result<Vec<_>, _>>()
+            .map_err(Into::into)
+    }
+
     #[allow(
         dead_code,
         reason = "some integration tests import the store without public HTTP routes"
@@ -3243,6 +3407,99 @@
     pub(crate) object_format: String,
     pub(crate) created_at: i64,
     pub(crate) archived_at: Option<i64>,
+}
+
+#[derive(Debug, Serialize)]
+pub(crate) struct AccountInspection {
+    #[serde(rename = "type")]
+    pub(crate) record_type: &'static str,
+    pub(crate) username: String,
+    pub(crate) administrator: bool,
+    pub(crate) state: String,
+    pub(crate) created_at: i64,
+    pub(crate) keys: Vec<KeyInspection>,
+}
+
+#[derive(Debug, Serialize)]
+pub(crate) struct KeyInspection {
+    pub(crate) label: String,
+    pub(crate) fingerprint: String,
+    pub(crate) created_at: i64,
+    pub(crate) last_used_at: Option<i64>,
+    pub(crate) revoked_at: Option<i64>,
+}
+
+#[derive(Debug, Serialize)]
+pub(crate) struct GitIntentInspection {
+    #[serde(rename = "type")]
+    pub(crate) record_type: &'static str,
+    pub(crate) id: String,
+    pub(crate) repository_path: String,
+    pub(crate) actor: String,
+    pub(crate) quarantine_path: String,
+    pub(crate) state: String,
+    pub(crate) pack_name: Option<String>,
+    pub(crate) created_at: i64,
+}
+
+#[derive(Debug, Serialize)]
+pub(crate) struct RepositoryInspection {
+    #[serde(rename = "type")]
+    pub(crate) record_type: &'static str,
+    pub(crate) id: String,
+    pub(crate) owner: String,
+    pub(crate) slug: String,
+    pub(crate) visibility: String,
+    pub(crate) state: String,
+    pub(crate) object_format: String,
+    pub(crate) created_at: i64,
+    pub(crate) archived_at: Option<i64>,
+}
+
+impl From<RepositoryRecord> for RepositoryInspection {
+    fn from(repository: RepositoryRecord) -> Self {
+        Self {
+            record_type: "repository",
+            id: repository.id,
+            owner: repository.owner,
+            slug: repository.slug,
+            visibility: repository.visibility,
+            state: repository.state,
+            object_format: repository.object_format,
+            created_at: repository.created_at,
+            archived_at: repository.archived_at,
+        }
+    }
+}
+
+#[derive(Debug, Serialize)]
+pub(crate) struct DumpRow {
+    #[serde(rename = "type")]
+    pub(crate) record_type: &'static str,
+    pub(crate) table: String,
+    pub(crate) columns: Vec<DumpColumn>,
+}
+
+#[derive(Debug, Serialize)]
+pub(crate) struct DumpColumn {
+    pub(crate) name: String,
+    pub(crate) value: DumpValue,
+}
+
+#[derive(Debug, Serialize)]
+#[serde(tag = "type", content = "value", rename_all = "kebab-case")]
+pub(crate) enum DumpValue {
+    Null,
+    Integer(i64),
+    Real(String),
+    TextUtf8(String),
+    TextHex(String),
+    BlobHex(String),
+}
+
+struct TableColumn {
+    name: String,
+    primary_key: i64,
 }
 
 #[allow(
@@ -4537,6 +4794,33 @@
     })
 }
 
+fn quote_identifier(identifier: &str) -> String {
+    format!("\"{}\"", identifier.replace('"', "\"\""))
+}
+
+fn dump_value(value: ValueRef<'_>) -> DumpValue {
+    match value {
+        ValueRef::Null => DumpValue::Null,
+        ValueRef::Integer(value) => DumpValue::Integer(value),
+        ValueRef::Real(value) => DumpValue::Real(format!("{value:.17e}")),
+        ValueRef::Text(value) => match std::str::from_utf8(value) {
+            Ok(value) => DumpValue::TextUtf8(value.to_owned()),
+            Err(_) => DumpValue::TextHex(hex(value)),
+        },
+        ValueRef::Blob(value) => DumpValue::BlobHex(hex(value)),
+    }
+}
+
+fn hex(bytes: &[u8]) -> String {
+    const HEX: &[u8; 16] = b"0123456789abcdef";
+    let mut output = String::with_capacity(bytes.len() * 2);
+    for byte in bytes {
+        output.push(char::from(HEX[(byte >> 4) as usize]));
+        output.push(char::from(HEX[(byte & 0x0f) as usize]));
+    }
+    output
+}
+
 fn is_unique_constraint(error: &rusqlite::Error) -> bool {
     matches!(
         error,
@@ -4552,12 +4836,7 @@
 )]
 pub(crate) fn doctor(instance_dir: &Path) -> Result<(), StoreError> {
     let path = instance_dir.join(DATABASE_FILE);
-    let connection = Connection::open_with_flags(
-        path,
-        OpenFlags::SQLITE_OPEN_READ_WRITE | OpenFlags::SQLITE_OPEN_NO_MUTEX,
-    )?;
-    configure(&connection)?;
-    let store = Store { connection };
+    let store = Store::open_read_only(&path)?;
     let actual = store.schema_version()?;
     if actual != SCHEMA_VERSION {
         return Err(StoreError::SchemaVersion {

@@ -85,6 +85,7 @@
         .execute_batch(CURRENT_DATABASE)
         .expect("create the current database");
     drop(database);
+    prepare_doctor_files(&instance);
 
     let output = instance.run(&[
         "--config",
@@ -92,7 +93,11 @@
         "doctor",
     ]);
 
-    assert!(output.status.success());
+    assert!(
+        output.status.success(),
+        "doctor failed: {}",
+        String::from_utf8_lossy(&output.stderr)
+    );
     assert!(output.stdout.is_empty());
     assert!(output.stderr.is_empty());
 }
@@ -146,6 +151,7 @@
         )
         .expect("create a foreign-key violation");
     drop(database);
+    prepare_doctor_files(&instance);
 
     let output = instance.run(&[
         "--config",
@@ -155,7 +161,24 @@
 
     assert_eq!(output.status.code(), Some(1));
     assert!(output.stdout.is_empty());
-    assert!(String::from_utf8_lossy(&output.stderr).contains("foreign key violation"));
+    assert!(
+        String::from_utf8_lossy(&output.stderr).contains("foreign key violation"),
+        "unexpected doctor error: {}",
+        String::from_utf8_lossy(&output.stderr)
+    );
+}
+
+fn prepare_doctor_files(instance: &TestInstance) {
+    fs::set_permissions(
+        instance.path().join("tit.sqlite3"),
+        fs::Permissions::from_mode(0o600),
+    )
+    .expect("make the database private");
+    let repositories = instance.path().join("repositories");
+    fs::create_dir(&repositories).expect("create the repository directory");
+    fs::set_permissions(&repositories, fs::Permissions::from_mode(0o700))
+        .expect("make the repository directory private");
+    create_ssh_key_fixture(&instance.path().join("ssh_host_ed25519_key"));
 }
 
 #[test]
@@ -685,6 +708,216 @@
         assert_eq!(rejected.status.code(), Some(1), "accepted {slug:?}");
         assert!(rejected.stdout.is_empty());
     }
+}
+
+#[test]
+fn doctor_inspect_and_dump_are_read_only_and_detect_operational_damage() {
+    let instance = TestInstance::new();
+    create_administrator(&instance, "alice");
+    let config = instance.config().to_str().expect("a UTF-8 path");
+    let created = instance.run(&[
+        "--config",
+        config,
+        "admin",
+        "repository",
+        "create",
+        "alice",
+        "project",
+    ]);
+    assert!(created.status.success());
+    create_ssh_key_fixture(&instance.path().join("ssh_host_ed25519_key"));
+
+    let doctor = instance.run(&["--config", config, "doctor"]);
+    assert!(
+        doctor.status.success(),
+        "doctor failed: {}",
+        String::from_utf8_lossy(&doctor.stderr)
+    );
+    assert!(doctor.stdout.is_empty());
+
+    let account = instance.run(&["--config", config, "inspect", "account", "alice"]);
+    assert!(account.status.success());
+    let account: serde_json::Value =
+        serde_json::from_slice(&account.stdout).expect("parse the account inspection");
+    assert_eq!(account["type"], "account");
+    assert_eq!(account["username"], "alice");
+    assert_eq!(account["keys"].as_array().map(Vec::len), Some(1));
+
+    let repository = instance.run(&[
+        "--config",
+        config,
+        "inspect",
+        "repository",
+        "alice",
+        "project",
+    ]);
+    assert!(repository.status.success());
+    let repository: serde_json::Value =
+        serde_json::from_slice(&repository.stdout).expect("parse the repository inspection");
+    assert_eq!(repository["type"], "repository");
+    assert_eq!(repository["object_format"], "sha1");
+    let repository_id = repository["id"]
+        .as_str()
+        .expect("read the repository identifier")
+        .to_owned();
+
+    let first_dump = instance.run(&["--config", config, "dump"]);
+    let second_dump = instance.run(&["--config", config, "dump"]);
+    assert!(first_dump.status.success());
+    assert_eq!(first_dump.stdout, second_dump.stdout);
+    let dump_rows = String::from_utf8(first_dump.stdout).expect("read the JSON Lines dump");
+    assert!(!dump_rows.is_empty());
+    for line in dump_rows.lines() {
+        let row: serde_json::Value = serde_json::from_str(line).expect("parse a dump row");
+        assert_eq!(row["type"], "sqlite-row");
+        assert!(row["table"].is_string());
+        assert!(row["columns"].is_array());
+    }
+    assert!(dump_rows.contains("\"type\":\"blob-hex\""));
+
+    let backup_directory = TempDir::new().expect("create a backup directory");
+    let backup = backup_directory.path().join("instance.tar");
+    let backup_output = instance.run(&[
+        "--config",
+        config,
+        "backup",
+        backup.to_str().expect("a UTF-8 backup path"),
+    ]);
+    assert!(backup_output.status.success());
+    let checked_backup = instance.run(&[
+        "--config",
+        config,
+        "doctor",
+        "--backup",
+        backup.to_str().expect("a UTF-8 backup path"),
+    ]);
+    assert!(checked_backup.status.success());
+
+    let database_path = instance.path().join("tit.sqlite3");
+    let database = rusqlite::Connection::open(&database_path).expect("open the database");
+    let repository_path = instance
+        .path()
+        .join("repositories")
+        .join(format!("{repository_id}.git"));
+    let intent_id = "11111111111111111111111111111111";
+    let quarantine = repository_path
+        .join("objects")
+        .join("tit-quarantine")
+        .join(intent_id);
+    database
+        .execute(
+            "INSERT INTO git_operation_intent
+             (id, repository_path, actor, initial_refs, proposed_refs, event_payload,
+              quarantine_path, state, pack_name, created_at)
+             VALUES (?1, ?2, 'alice', X'', X'', X'', ?3, 'pending', NULL, 1)",
+            rusqlite::params![
+                intent_id,
+                repository_path.to_str().expect("a UTF-8 repository path"),
+                quarantine.to_str().expect("a UTF-8 quarantine path")
+            ],
+        )
+        .expect("create an incomplete intent");
+    drop(database);
+
+    let incomplete = instance.run(&["--config", config, "doctor"]);
+    assert_eq!(incomplete.status.code(), Some(1));
+    assert!(String::from_utf8_lossy(&incomplete.stderr).contains("incomplete intents"));
+    let intent = instance.run(&["--config", config, "inspect", "intent", intent_id]);
+    assert!(intent.status.success());
+    let intent: serde_json::Value =
+        serde_json::from_slice(&intent.stdout).expect("parse the intent inspection");
+    assert_eq!(intent["state"], "pending");
+    let database = rusqlite::Connection::open(&database_path).expect("reopen the database");
+    assert_eq!(
+        database
+            .query_row(
+                "SELECT state FROM git_operation_intent WHERE id = ?1",
+                [intent_id],
+                |row| row.get::<_, String>(0),
+            )
+            .expect("read the intent state"),
+        "pending"
+    );
+    drop(database);
+    let repaired_intent = instance.run(&["--config", config, "repair", "intents"]);
+    assert!(
+        repaired_intent.status.success(),
+        "intent repair failed: {}",
+        String::from_utf8_lossy(&repaired_intent.stderr)
+    );
+    let database = rusqlite::Connection::open(&database_path).expect("reopen the database");
+    assert_eq!(
+        database
+            .query_row(
+                "SELECT state FROM git_operation_intent WHERE id = ?1",
+                [intent_id],
+                |row| row.get::<_, String>(0),
+            )
+            .expect("read the repaired intent state"),
+        "abandoned"
+    );
+    drop(database);
+
+    fs::create_dir_all(&quarantine).expect("create quarantine debris");
+    fs::write(quarantine.join("incoming.pack"), b"debris").expect("write quarantine debris");
+    let debris = instance.run(&["--config", config, "doctor"]);
+    assert_eq!(debris.status.code(), Some(1));
+    assert!(String::from_utf8_lossy(&debris.stderr).contains("quarantine debris"));
+    let repaired_quarantine = instance.run(&["--config", config, "repair", "quarantine"]);
+    assert!(
+        repaired_quarantine.status.success(),
+        "quarantine repair failed: {}",
+        String::from_utf8_lossy(&repaired_quarantine.stderr)
+    );
+
+    let database = rusqlite::Connection::open(&database_path).expect("reopen the database");
+    database
+        .execute_batch("DROP INDEX audit_event_history")
+        .expect("remove a required index");
+    drop(database);
+    let missing_index = instance.run(&["--config", config, "doctor"]);
+    assert_eq!(missing_index.status.code(), Some(1));
+    assert!(String::from_utf8_lossy(&missing_index.stderr).contains("audit_event_history"));
+    let database = rusqlite::Connection::open(&database_path).expect("reopen the database");
+    database
+        .execute_batch(
+            "CREATE INDEX audit_event_history
+             ON audit_event (created_at DESC, id DESC)",
+        )
+        .expect("restore the required index");
+    drop(database);
+
+    let bad_ref = repository_path.join("refs").join("heads").join("broken");
+    fs::create_dir_all(bad_ref.parent().expect("the ref has a parent"))
+        .expect("create the ref directory");
+    fs::write(&bad_ref, b"deadbeef\n").expect("damage a Git ref");
+    let damaged_git = instance.run(&["--config", config, "doctor"]);
+    assert_eq!(damaged_git.status.code(), Some(1));
+    fs::remove_file(&bad_ref).expect("remove the damaged ref");
+
+    fs::set_permissions(instance.config(), fs::Permissions::from_mode(0o644))
+        .expect("make the configuration unsafe");
+    let unsafe_permissions = instance.run(&["--config", config, "doctor"]);
+    assert_eq!(unsafe_permissions.status.code(), Some(1));
+    assert!(String::from_utf8_lossy(&unsafe_permissions.stderr).contains("owner-only"));
+    fs::set_permissions(instance.config(), fs::Permissions::from_mode(0o600))
+        .expect("restore the configuration permissions");
+
+    let mut backup_bytes = fs::read(&backup).expect("read the backup");
+    let offset = backup_bytes
+        .windows(b"version = 1".len())
+        .position(|candidate| candidate == b"version = 1")
+        .expect("find archived configuration data");
+    backup_bytes[offset] ^= 1;
+    fs::write(&backup, backup_bytes).expect("damage the backup");
+    let damaged_backup = instance.run(&[
+        "--config",
+        config,
+        "doctor",
+        "--backup",
+        backup.to_str().expect("a UTF-8 backup path"),
+    ]);
+    assert_eq!(damaged_backup.status.code(), Some(1));
 }
 
 fn create_administrator(instance: &TestInstance, username: &str) {

@@ -1,5 +1,6 @@
 use std::fs;
 use std::net::{SocketAddr, TcpListener};
+use std::os::unix::fs::PermissionsExt;
 use std::path::{Path, PathBuf};
 use std::process::{Command, Output};
 
@@ -19,6 +20,10 @@
             "version = 1\npublic_url = \"https://tit.example/\"\n",
         )
         .expect("write the configuration");
+        fs::set_permissions(directory.path(), fs::Permissions::from_mode(0o700))
+            .expect("make the instance directory private");
+        fs::set_permissions(&config, fs::Permissions::from_mode(0o600))
+            .expect("make the configuration private");
         Self { directory, config }
     }