diff --git a/README.md b/README.md index 99a2476b814eaac69562c857b7b0a1b7b82f4120..0230d5df636e4eea1b5a3a2300e0d117e1f4fe73 100644 --- a/README.md +++ b/README.md @@ -105,6 +105,53 @@ It does not activate the restored instance. To activate it, stop the old server and explicitly start `tit --config /srv/tit-restored/config.toml serve`. +## Diagnostics and repair + +Run the read-only instance checks with: + +```text +tit --config /srv/tit/config.toml doctor +``` + +Also check one or more backup archives with: + +```text +tit --config /srv/tit/config.toml doctor --backup /var/backups/tit-2026-07-23.tar +``` + +Doctor checks configuration, private permissions, the schema, record +relations, indexes, incomplete intents, Git refs and reachable objects, +quarantine debris, the SSH host key, and each supplied backup. It does not +change the instance. + +Use the separate repair commands only after you review the doctor error: + +```text +tit --config /srv/tit/config.toml repair intents +tit --config /srv/tit/config.toml repair quarantine +``` + +Stop the server before repair. Intent repair uses the normal recovery rules. +Quarantine repair refuses to run while an incomplete intent exists. + +Inspect one typed record as JSON: + +```text +tit --config /srv/tit/config.toml inspect account alice +tit --config /srv/tit/config.toml inspect repository alice example +tit --config /srv/tit/config.toml inspect intent 0123456789abcdef0123456789abcdef +``` + +Write all SQLite rows as deterministic JSON Lines: + +```text +tit --config /srv/tit/config.toml dump >tit-dump.jsonl +``` + +The dump can contain credential hashes, session hashes, token hashes, and SSH +public keys. Store it as a secret. The dump is for inspection and comparison; +it is not a restore format. + An authenticated account can create a repository with SSH: ```text @@ -540,3 +587,18 @@ database validation, and restored Git object access. Read the [backup and restore architectural decision record](docs/adr/0029-backup-and-restore.md) for the archive, gate, credential, and activation contracts. + +## Milestone 6.3 gate + +Install stock Git and stock `ssh-keygen`. Then, run the diagnostics gate: + +```text +./scripts/check-m6-3 +``` + +This command tests read-only doctor checks, explicit repair, typed inspection, +deterministic JSON Lines output, damaged Git state, incomplete intents, +quarantine debris, missing indexes, unsafe permissions, and changed backup +data. Read the +[read-only diagnostics architectural decision record](docs/adr/0030-read-only-diagnostics.md) +for the check, repair, inspection, and dump contracts. diff --git a/docs/adr/0030-read-only-diagnostics.md b/docs/adr/0030-read-only-diagnostics.md new file mode 100644 index 0000000000000000000000000000000000000000..6881e40b810427d53bde78a0d617bd3672bcd66a --- /dev/null +++ b/docs/adr/0030-read-only-diagnostics.md @@ -1,0 +1,95 @@ +# Architectural decision record 0030: read-only diagnostics + +Status: Accepted + +Date: 2026-07-23 + +## Context + +The initial `tit doctor` command checked only the SQLite schema version, +database pages, and foreign keys. It opened a read-write connection and applied +the normal connection configuration. It did not check the filesystem, Git +state, incomplete cross-store operations, or backup archives. + +An operator also needed stable ways to examine records. Direct SQLite queries +require knowledge of the schema and can accidentally change the database. + +## Decision + +`tit doctor` opens SQLite with `SQLITE_OPEN_READ_ONLY`. It does not take the +instance lock, migrate a schema, recover an intent, remove quarantine data, or +create a file. It checks: + +- the instance, configuration, database, repository-root, and SSH host-key + types and permissions; +- the current schema version, SQLite integrity, foreign keys, and all required + indexes; +- incomplete Git operation intents and pull-request ref intents; +- the database record for each repository and each repository directory; +- the Git object format, refs, and all reachable Git objects; +- unknown repository-root entries and quarantine debris; and +- each backup archive supplied with `--backup`. + +The configuration parser runs before these checks, so configuration syntax and +values are part of the doctor result. A missing SSH host key is an error because +the instance is not ready to preserve its SSH identity. + +Add separate `repair intents` and `repair quarantine` commands. Each repair +command takes the instance lock. Intent repair uses the normal recovery logic. +Quarantine repair refuses to run while an incomplete intent exists. Doctor +never calls either repair command. + +Add typed `inspect account`, `inspect repository`, and `inspect intent` +commands. Each command returns one version-independent JSON object for the +selected record type. Account inspection does not return the recovery hash or +canonical public-key text. + +Add `tit dump`. It streams one JSON object for each SQLite row in deterministic +table and primary-key order. Each object identifies the table and gives each +column name, SQLite value type, and value. BLOB values and invalid UTF-8 text +use lowercase hexadecimal. Valid UTF-8 text stays text. Real values use a +deterministic 17-digit exponential form. + +The dump is a raw operational artifact. It can contain credential hashes, +session hashes, feed-token hashes, invitation hashes, and SSH public keys. +Operators must store it as a secret. + +## Failure and threat cases + +Opening a normal `Store` can migrate an old schema and can create a migration +backup. Diagnostics use a separate read-only constructor, so a doctor, +inspection, or dump cannot do these operations. + +Incomplete intents and quarantine directories can be necessary for recovery. +Doctor reports them and leaves them unchanged. The explicit repair commands +serialize with the server through the instance lock. + +A repository directory can be a symbolic link or can have an identifier that +has no database record. Doctor rejects both cases. It also opens the repository +without a runtime Git command and walks reachable objects with the established +object-count limit. + +A damaged backup can change between validation passes. Backup validation checks +the manifest entry type, size, and checksum. Restore checks them again while it +extracts each file. + +## Evidence + +The CLI tests run doctor on a correct instance and on instances with a foreign +key violation, a missing index, an incomplete intent, quarantine debris, an +invalid Git ref, unsafe permissions, and a changed backup. The test confirms +that doctor leaves the incomplete intent unchanged. It then runs each explicit +repair command and checks the result. + +The same test checks typed account, repository, and intent JSON. It runs the +JSON Lines dump two times and checks byte-for-byte equality and valid typed +rows. + +## Consequences + +Doctor can run while the server is active because it does not change state. +The result is a point-in-time sequence of checks, not a global snapshot. Use an +online backup when one coherent cross-store snapshot is necessary. + +The JSON Lines dump exposes storage details intentionally. It is suitable for +external inspection and comparison, but it is not a restore format. diff --git a/scripts/check-m6-3 b/scripts/check-m6-3 new file mode 100755 index 0000000000000000000000000000000000000000..b3a82c6d5cc16ed7d6ae148158e0103766e789ff --- /dev/null +++ b/scripts/check-m6-3 @@ -1,0 +1,5 @@ +#!/bin/sh +set -eu + +./scripts/check +cargo test --locked --release --test cli doctor_ diff --git a/src/backup.rs b/src/backup.rs index a146d339519295d2f7e0714f9f121600e6fa732b..9caba392fa2c0509231466383dbaa6b25867315d 100644 --- a/src/backup.rs +++ b/src/backup.rs @@ -86,6 +86,13 @@ Ok(()) } +pub(crate) fn check_archive(archive_path: &Path) -> Result<(), BackupError> { + validate_absolute_clean(archive_path)?; + let manifest = read_manifest(archive_path)?; + let expected = expected_files(&manifest)?; + verify_archive(archive_path, &expected) +} + fn create_archive( instance_dir: &Path, config_path: &Path, diff --git a/src/cli.rs b/src/cli.rs index 6077cfd94df95de814e87428850b833b8ae06948..28c8a816c43ca5ab837886c8424337ff50754b23 100644 --- a/src/cli.rs +++ b/src/cli.rs @@ -50,8 +50,24 @@ Serve, /// Create a single-use signup invitation InviteCode, - /// Check the instance database - Doctor, + /// Check the instance without changing it + Doctor { + /// Also check the manifest and checksums in FILE + #[arg(long = "backup", value_name = "FILE")] + backups: Vec, + }, + /// Show one typed instance record as JSON + Inspect { + #[command(subcommand)] + command: InspectCommand, + }, + /// Write all SQLite rows as deterministic JSON Lines + Dump, + /// Run an explicit repair operation + Repair { + #[command(subcommand)] + command: RepairCommand, + }, /// Create a backup archive Backup { /// Write the backup archive to FILE @@ -74,6 +90,24 @@ #[command(subcommand)] command: AdminCommand, }, +} + +#[derive(Clone, Debug, Subcommand)] +pub(crate) enum RepairCommand { + /// Recover incomplete Git and pull-request ref intents + Intents, + /// Remove quarantine debris after all intents are complete + Quarantine, +} + +#[derive(Clone, Debug, Subcommand)] +pub(crate) enum InspectCommand { + /// Show an account and its SSH key metadata + Account { username: String }, + /// Show a repository record after Git validation + Repository { owner: String, slug: String }, + /// Show a Git operation intent + Intent { id: String }, } #[derive(Clone, Debug, Subcommand)] diff --git a/src/diagnostics.rs b/src/diagnostics.rs new file mode 100644 index 0000000000000000000000000000000000000000..d174dcc8980e610e1619192b41438001875b0c16 --- /dev/null +++ b/src/diagnostics.rs @@ -1,0 +1,263 @@ +use std::collections::HashSet; +use std::fs; +use std::os::unix::fs::PermissionsExt; +use std::path::{Path, PathBuf}; + +use thiserror::Error; + +use crate::backup::{self, BackupError}; +use crate::config::Config; +use crate::git::repository::{GitRepository, GitRepositoryError}; +use crate::instance::REPOSITORY_DIRECTORY; +use crate::serve::{self, ServeError}; +use crate::store::{ + AccountInspection, DATABASE_FILE, DumpRow, GitIntentInspection, RepositoryInspection, Store, + StoreError, +}; + +const REQUIRED_INDEXES: &[&str] = &[ + "audit_event_correlation", + "audit_event_history", + "feed_token_account_active", + "feed_token_repository_active", + "git_operation_intent_incomplete", + "issue_assignee_account", + "issue_comment_history", + "issue_label_label", + "issue_repository_state", + "label_repository_name", + "login_nonce_active", + "m1a_child_state_parent", + "m1a_parent_created_at", + "pull_request_ref_intent_incomplete", + "pull_request_repository_state", + "pull_request_review_status", + "pull_request_review_timeline", + "pull_request_revision_history", + "repository_collaborator_account", + "repository_event_feed", + "repository_event_issue_timeline", + "repository_event_pull_request_timeline", + "signup_invitation_active", + "ssh_public_key_account", + "watch_account_activity", + "web_session_account_active", +]; + +pub(crate) fn doctor(config: &Config, backups: &[PathBuf]) -> Result<(), DiagnosticError> { + check_private_directory(&config.instance_dir)?; + check_private_file(&config.config_path)?; + let database = config.instance_dir.join(DATABASE_FILE); + check_private_file(&database)?; + crate::store::doctor(&config.instance_dir)?; + + let store = Store::open_read_only(&database)?; + let indexes: HashSet<_> = store.index_names()?.into_iter().collect(); + for required in REQUIRED_INDEXES { + if !indexes.contains(*required) { + return Err(DiagnosticError::MissingIndex((*required).to_owned())); + } + } + let git_intents = store.incomplete_git_intents()?; + let pull_request_intents = store.incomplete_pull_request_ref_intents()?; + if !git_intents.is_empty() || !pull_request_intents.is_empty() { + return Err(DiagnosticError::IncompleteIntents { + git: git_intents.len(), + pull_request: pull_request_intents.len(), + }); + } + check_repositories(&config.instance_dir, &store)?; + serve::check_host_key(&config.instance_dir)?; + for archive in backups { + backup::check_archive(archive)?; + } + Ok(()) +} + +pub(crate) fn inspect_account( + config: &Config, + username: &str, +) -> Result { + let store = Store::open_read_only(&config.instance_dir.join(DATABASE_FILE))?; + Ok(store.inspect_account(username)?) +} + +pub(crate) fn inspect_repository( + config: &Config, + owner: &str, + slug: &str, +) -> Result { + let store = Store::open_read_only(&config.instance_dir.join(DATABASE_FILE))?; + let repository = store.repository(owner, slug)?; + check_repository(&config.instance_dir, &repository)?; + Ok(repository.into()) +} + +pub(crate) fn inspect_intent( + config: &Config, + id: &str, +) -> Result { + let store = Store::open_read_only(&config.instance_dir.join(DATABASE_FILE))?; + Ok(store.inspect_git_intent(id)?) +} + +pub(crate) fn dump( + config: &Config, + visit: impl FnMut(DumpRow) -> bool, +) -> Result<(), DiagnosticError> { + let store = Store::open_read_only(&config.instance_dir.join(DATABASE_FILE))?; + Ok(store.dump_rows(visit)?) +} + +fn check_repositories(instance_dir: &Path, store: &Store) -> Result<(), DiagnosticError> { + let root = instance_dir.join(REPOSITORY_DIRECTORY); + check_private_directory(&root)?; + let repositories = store.all_repositories()?; + let expected: HashSet<_> = repositories + .iter() + .map(|repository| format!("{}.git", repository.id)) + .collect(); + for repository in &repositories { + check_repository(instance_dir, repository)?; + } + for entry in fs::read_dir(&root).map_err(|source| DiagnosticError::Io { + path: root.clone(), + source, + })? { + let entry = entry.map_err(|source| DiagnosticError::Io { + path: root.clone(), + source, + })?; + let Some(name) = entry.file_name().to_str().map(str::to_owned) else { + return Err(DiagnosticError::UnknownRepositoryEntry(entry.path())); + }; + if !expected.contains(&name) { + return Err(DiagnosticError::UnknownRepositoryEntry(entry.path())); + } + } + Ok(()) +} + +fn check_repository( + instance_dir: &Path, + repository: &crate::store::RepositoryRecord, +) -> Result<(), DiagnosticError> { + let path = instance_dir + .join(REPOSITORY_DIRECTORY) + .join(format!("{}.git", repository.id)); + let metadata = fs::symlink_metadata(&path).map_err(|source| DiagnosticError::Io { + path: path.clone(), + source, + })?; + if metadata.file_type().is_symlink() || !metadata.file_type().is_dir() { + return Err(DiagnosticError::UnsafePath(path)); + } + let git = GitRepository::open(&path)?; + let format = match git.object_format() { + gix::hash::Kind::Sha1 => "sha1", + gix::hash::Kind::Sha256 => "sha256", + _ => return Err(DiagnosticError::UnsupportedObjectFormat), + }; + if format != repository.object_format { + return Err(DiagnosticError::ObjectFormat(repository.id.clone())); + } + git.integrity_check()?; + + let quarantine = path.join("objects").join("tit-quarantine"); + match fs::symlink_metadata(&quarantine) { + Ok(metadata) => { + if metadata.file_type().is_symlink() || !metadata.file_type().is_dir() { + return Err(DiagnosticError::UnsafePath(quarantine)); + } + if fs::read_dir(&quarantine) + .map_err(|source| DiagnosticError::Io { + path: quarantine.clone(), + source, + })? + .next() + .is_some() + { + return Err(DiagnosticError::QuarantineDebris(quarantine)); + } + } + Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} + Err(source) => { + return Err(DiagnosticError::Io { + path: quarantine, + source, + }); + } + } + Ok(()) +} + +fn check_private_directory(path: &Path) -> Result<(), DiagnosticError> { + let metadata = fs::symlink_metadata(path).map_err(|source| DiagnosticError::Io { + path: path.to_owned(), + source, + })?; + if metadata.file_type().is_symlink() || !metadata.file_type().is_dir() { + return Err(DiagnosticError::UnsafePath(path.to_owned())); + } + let mode = metadata.permissions().mode() & 0o777; + if mode & 0o077 != 0 { + return Err(DiagnosticError::Permissions { + path: path.to_owned(), + mode, + }); + } + Ok(()) +} + +fn check_private_file(path: &Path) -> Result<(), DiagnosticError> { + let metadata = fs::symlink_metadata(path).map_err(|source| DiagnosticError::Io { + path: path.to_owned(), + source, + })?; + if metadata.file_type().is_symlink() || !metadata.file_type().is_file() { + return Err(DiagnosticError::UnsafePath(path.to_owned())); + } + let mode = metadata.permissions().mode() & 0o777; + if mode & 0o077 != 0 { + return Err(DiagnosticError::Permissions { + path: path.to_owned(), + mode, + }); + } + Ok(()) +} + +#[derive(Debug, Error)] +pub(crate) enum DiagnosticError { + #[error("diagnostic path is unsafe: {0}")] + UnsafePath(PathBuf), + #[error("diagnostic path permissions for {path} are {mode:o}, expected owner-only access")] + Permissions { path: PathBuf, mode: u32 }, + #[error("required database index does not exist: {0}")] + MissingIndex(String), + #[error( + "incomplete intents exist: {git} Git operation intents and {pull_request} pull-request ref intents" + )] + IncompleteIntents { git: usize, pull_request: usize }, + #[error("repository directory has an unknown entry: {0}")] + UnknownRepositoryEntry(PathBuf), + #[error("repository has quarantine debris: {0}")] + QuarantineDebris(PathBuf), + #[error("repository object format does not match its database record: {0}")] + ObjectFormat(String), + #[error("repository object format is not supported")] + UnsupportedObjectFormat, + #[error("diagnostic I/O failed for {path}: {source}")] + Io { + path: PathBuf, + source: std::io::Error, + }, + #[error(transparent)] + Store(#[from] StoreError), + #[error(transparent)] + Git(#[from] GitRepositoryError), + #[error(transparent)] + HostKey(#[from] ServeError), + #[error(transparent)] + Backup(#[from] BackupError), +} diff --git a/src/main.rs b/src/main.rs index 00df94cf7e7c359534b047d99e8bbd4bffa24a32..7a8922efeb133f9b29add75bce9806e93a2ee2e7 100644 --- a/src/main.rs +++ b/src/main.rs @@ -10,6 +10,7 @@ mod cli; mod config; mod control; +mod diagnostics; mod domain; mod feed; mod feed_token; @@ -23,6 +24,7 @@ mod markdown; mod policy; mod pull_request; +mod repair; mod repository; mod search; mod serve; @@ -38,8 +40,8 @@ use clap::Parser; use crate::cli::{ - AccountCommand, AdminCommand, Cli, CollaboratorRole, Command, ObjectFormat, RepositoryCommand, - RepositoryVisibility, SetupCommand, + AccountCommand, AdminCommand, Cli, CollaboratorRole, Command, InspectCommand, ObjectFormat, + RepairCommand, RepositoryCommand, RepositoryVisibility, SetupCommand, }; #[tokio::main] @@ -82,13 +84,28 @@ } } } - Some(Command::Doctor) => match store::doctor(&config.instance_dir) { + Some(Command::Doctor { backups }) => match diagnostics::doctor(&config, &backups) { Ok(()) => ExitCode::SUCCESS, Err(error) => { eprintln!("tit: {error}"); ExitCode::FAILURE } }, + Some(Command::Inspect { command }) => run_inspect_command(&config, command), + Some(Command::Dump) => run_dump_command(&config), + Some(Command::Repair { command }) => { + let result = match command { + RepairCommand::Intents => repair::intents(&config.instance_dir), + RepairCommand::Quarantine => repair::quarantine(&config.instance_dir), + }; + match result { + Ok(()) => ExitCode::SUCCESS, + Err(error) => { + eprintln!("tit: {error}"); + ExitCode::FAILURE + } + } + } Some(Command::Backup { output }) => run_backup(&config, &output).await, Some(Command::Restore { .. }) => { unreachable!("the restore command runs before configuration is loaded") @@ -129,6 +146,71 @@ command: AdminCommand::Audit { limit }, }) => run_audit_command(&config.instance_dir, limit), }, + Err(error) => { + eprintln!("tit: {error}"); + ExitCode::FAILURE + } + } +} + +fn run_inspect_command(config: &config::Config, command: InspectCommand) -> ExitCode { + let result = match command { + InspectCommand::Account { username } => { + serialize_inspection(diagnostics::inspect_account(config, &username)) + } + InspectCommand::Repository { owner, slug } => { + serialize_inspection(diagnostics::inspect_repository(config, &owner, &slug)) + } + InspectCommand::Intent { id } => { + serialize_inspection(diagnostics::inspect_intent(config, &id)) + } + }; + match result { + Ok(line) => match writeln!(io::stdout().lock(), "{line}") { + Ok(()) => ExitCode::SUCCESS, + Err(error) => { + eprintln!("tit: cannot write inspect information: {error}"); + ExitCode::FAILURE + } + }, + Err(error) => { + eprintln!("tit: {error}"); + ExitCode::FAILURE + } + } +} + +fn serialize_inspection( + result: Result, +) -> Result> { + Ok(serde_json::to_string(&result?)?) +} + +fn run_dump_command(config: &config::Config) -> ExitCode { + let result = (|| -> Result<(), Box> { + let mut output = io::stdout().lock(); + let mut output_error = None; + diagnostics::dump(config, |row| { + let result = serde_json::to_writer(&mut output, &row) + .map_err(|error| Box::new(error) as Box) + .and_then(|()| { + writeln!(output).map_err(|error| Box::new(error) as Box) + }); + match result { + Ok(()) => true, + Err(error) => { + output_error = Some(error); + false + } + } + })?; + if let Some(error) = output_error { + return Err(error); + } + Ok(()) + })(); + match result { + Ok(()) => ExitCode::SUCCESS, Err(error) => { eprintln!("tit: {error}"); ExitCode::FAILURE diff --git a/src/repair.rs b/src/repair.rs new file mode 100644 index 0000000000000000000000000000000000000000..fb569f18679dac8821eba72738f12d54a215f3d5 --- /dev/null +++ b/src/repair.rs @@ -1,0 +1,69 @@ +use std::fs; +use std::path::{Path, PathBuf}; + +use thiserror::Error; + +use crate::instance::{InstanceError, InstanceLock, REPOSITORY_DIRECTORY}; +use crate::pull_request::{PullRequestError, PullRequestService}; +use crate::store::{DATABASE_FILE, Store, StoreError}; + +pub(crate) fn intents(instance_dir: &Path) -> Result<(), RepairError> { + let _lock = InstanceLock::acquire(instance_dir)?; + crate::store::doctor(instance_dir)?; + let database = instance_dir.join(DATABASE_FILE); + let repositories = instance_dir.join(REPOSITORY_DIRECTORY); + PullRequestService::new(&database, &repositories).recover()?; + Ok(()) +} + +pub(crate) fn quarantine(instance_dir: &Path) -> Result<(), RepairError> { + let _lock = InstanceLock::acquire(instance_dir)?; + crate::store::doctor(instance_dir)?; + let database = instance_dir.join(DATABASE_FILE); + let store = Store::open_read_only(&database)?; + if !store.incomplete_git_intents()?.is_empty() + || !store.incomplete_pull_request_ref_intents()?.is_empty() + { + return Err(RepairError::IncompleteIntents); + } + let repositories = store.all_repositories()?; + drop(store); + for repository in repositories { + let path = instance_dir + .join(REPOSITORY_DIRECTORY) + .join(format!("{}.git", repository.id)) + .join("objects") + .join("tit-quarantine"); + match fs::symlink_metadata(&path) { + Ok(metadata) if metadata.file_type().is_dir() && !metadata.file_type().is_symlink() => { + fs::remove_dir_all(&path).map_err(|source| RepairError::Io { + path: path.clone(), + source, + })?; + } + Ok(_) => return Err(RepairError::UnsafePath(path)), + Err(error) if error.kind() == std::io::ErrorKind::NotFound => {} + Err(source) => return Err(RepairError::Io { path, source }), + } + } + Ok(()) +} + +#[derive(Debug, Error)] +pub(crate) enum RepairError { + #[error("repair path is unsafe: {0}")] + UnsafePath(PathBuf), + #[error("repair of quarantine debris requires all intents to be complete")] + IncompleteIntents, + #[error("repair I/O failed for {path}: {source}")] + Io { + path: PathBuf, + source: std::io::Error, + }, + #[error(transparent)] + Instance(#[from] InstanceError), + #[error(transparent)] + Store(#[from] StoreError), + #[error(transparent)] + PullRequest(#[from] PullRequestError), +} diff --git a/src/serve.rs b/src/serve.rs index b162eeada8a0cf3392f5a277f82acf60ae57d600..88484a7317e686e924bbbd0cb8471337be89dec3 100644 --- a/src/serve.rs +++ b/src/serve.rs @@ -145,6 +145,19 @@ read_host_key(&path) } +pub(crate) fn check_host_key(instance_dir: &Path) -> Result<(), ServeError> { + let path = instance_dir.join(HOST_KEY_FILE); + let metadata = fs::symlink_metadata(&path).map_err(|source| ServeError::HostKeyIo { + path: path.clone(), + source, + })?; + if metadata.file_type().is_symlink() || !metadata.file_type().is_file() { + return Err(ServeError::InvalidHostKeyFile(path)); + } + read_host_key(&path)?; + Ok(()) +} + fn create_host_key(path: &Path) -> Result { let key = PrivateKey::random(&mut rng(), Algorithm::Ed25519)?; let encoded = key.to_openssh(LineEnding::LF)?; diff --git a/src/store/mod.rs b/src/store/mod.rs index 88fac9733a7f7248c95e8fc2b735566225e3ec30..82e53b0820df06c26a32a6687a695260b4291899 100644 --- a/src/store/mod.rs +++ b/src/store/mod.rs @@ -4,7 +4,9 @@ use rusqlite::OpenFlags; use rusqlite::backup::Backup; +use rusqlite::types::ValueRef; use rusqlite::{Connection, OptionalExtension, TransactionBehavior}; +use serde::Serialize; use thiserror::Error; mod event; @@ -226,6 +228,16 @@ pub(crate) fn open_unmigrated(path: &Path) -> Result { let connection = Connection::open(path)?; configure(&connection)?; + Ok(Self { connection }) + } + + pub(crate) fn open_read_only(path: &Path) -> Result { + let connection = Connection::open_with_flags( + path, + OpenFlags::SQLITE_OPEN_READ_ONLY | OpenFlags::SQLITE_OPEN_NO_MUTEX, + )?; + connection.busy_timeout(BUSY_TIMEOUT)?; + connection.pragma_update(None, "foreign_keys", true)?; Ok(Self { connection }) } @@ -2933,6 +2945,158 @@ .map_err(Into::into) } + pub(crate) fn index_names(&self) -> Result, StoreError> { + let mut statement = self.connection.prepare( + "SELECT name FROM sqlite_schema + WHERE type = 'index' AND name NOT LIKE 'sqlite_%' + ORDER BY name", + )?; + statement + .query_map([], |row| row.get(0))? + .collect::, _>>() + .map_err(Into::into) + } + + pub(crate) fn inspect_account(&self, username: &str) -> Result { + let account = self + .connection + .query_row( + "SELECT id, username, is_administrator, state, created_at + FROM account WHERE username = ?1", + [username], + |row| { + Ok(( + row.get::<_, i64>(0)?, + row.get::<_, String>(1)?, + row.get::<_, bool>(2)?, + row.get::<_, String>(3)?, + row.get::<_, i64>(4)?, + )) + }, + ) + .optional()? + .ok_or_else(|| StoreError::AccountNotFound(username.to_owned()))?; + let mut statement = self.connection.prepare( + "SELECT label, fingerprint, created_at, last_used_at, revoked_at + FROM ssh_public_key WHERE account_id = ?1 ORDER BY id", + )?; + let keys = statement + .query_map([account.0], |row| { + Ok(KeyInspection { + label: row.get(0)?, + fingerprint: row.get(1)?, + created_at: row.get(2)?, + last_used_at: row.get(3)?, + revoked_at: row.get(4)?, + }) + })? + .collect::, _>>()?; + Ok(AccountInspection { + record_type: "account", + username: account.1, + administrator: account.2, + state: account.3, + created_at: account.4, + keys, + }) + } + + pub(crate) fn inspect_git_intent(&self, id: &str) -> Result { + self.connection + .query_row( + "SELECT id, repository_path, actor, quarantine_path, state, pack_name, created_at + FROM git_operation_intent WHERE id = ?1", + [id], + |row| { + Ok(GitIntentInspection { + record_type: "git-intent", + id: row.get(0)?, + repository_path: row.get(1)?, + actor: row.get(2)?, + quarantine_path: row.get(3)?, + state: row.get(4)?, + pack_name: row.get(5)?, + created_at: row.get(6)?, + }) + }, + ) + .optional()? + .ok_or_else(|| StoreError::Integrity(format!("Git intent does not exist: {id}"))) + } + + pub(crate) fn dump_rows( + &self, + mut visit: impl FnMut(DumpRow) -> bool, + ) -> Result<(), StoreError> { + let mut tables_statement = self.connection.prepare( + "SELECT name FROM sqlite_schema + WHERE type = 'table' AND name NOT LIKE 'sqlite_%' + ORDER BY name", + )?; + let tables = tables_statement + .query_map([], |row| row.get::<_, String>(0))? + .collect::, _>>()?; + for table in tables { + let columns = self.table_columns(&table)?; + let order = columns + .iter() + .filter(|column| column.primary_key > 0) + .map(|column| (column.primary_key, quote_identifier(&column.name))) + .collect::>(); + let order = if order.is_empty() { + "rowid".to_owned() + } else { + let mut order = order; + order.sort_by_key(|item| item.0); + order + .into_iter() + .map(|item| item.1) + .collect::>() + .join(", ") + }; + let sql = format!( + "SELECT * FROM {} ORDER BY {order}", + quote_identifier(&table) + ); + let mut statement = self.connection.prepare(&sql)?; + let mut rows = statement.query([])?; + while let Some(row) = rows.next()? { + let mut values = Vec::with_capacity(columns.len()); + for (index, column) in columns.iter().enumerate() { + values.push(DumpColumn { + name: column.name.clone(), + value: dump_value(row.get_ref(index)?), + }); + } + let output = DumpRow { + record_type: "sqlite-row", + table: table.clone(), + columns: values, + }; + if !visit(output) { + return Ok(()); + } + } + } + Ok(()) + } + + fn table_columns(&self, table: &str) -> Result, StoreError> { + let mut statement = self.connection.prepare( + "SELECT name, pk FROM pragma_table_info(?1) + ORDER BY cid", + )?; + statement + .query_map([table], |row| { + Ok(TableColumn { + name: row.get(0)?, + primary_key: row.get(1)?, + }) + })? + .collect::, _>>() + .map_err(Into::into) + } + #[allow( dead_code, reason = "some integration tests import the store without public HTTP routes" @@ -3243,6 +3407,99 @@ pub(crate) object_format: String, pub(crate) created_at: i64, pub(crate) archived_at: Option, +} + +#[derive(Debug, Serialize)] +pub(crate) struct AccountInspection { + #[serde(rename = "type")] + pub(crate) record_type: &'static str, + pub(crate) username: String, + pub(crate) administrator: bool, + pub(crate) state: String, + pub(crate) created_at: i64, + pub(crate) keys: Vec, +} + +#[derive(Debug, Serialize)] +pub(crate) struct KeyInspection { + pub(crate) label: String, + pub(crate) fingerprint: String, + pub(crate) created_at: i64, + pub(crate) last_used_at: Option, + pub(crate) revoked_at: Option, +} + +#[derive(Debug, Serialize)] +pub(crate) struct GitIntentInspection { + #[serde(rename = "type")] + pub(crate) record_type: &'static str, + pub(crate) id: String, + pub(crate) repository_path: String, + pub(crate) actor: String, + pub(crate) quarantine_path: String, + pub(crate) state: String, + pub(crate) pack_name: Option, + pub(crate) created_at: i64, +} + +#[derive(Debug, Serialize)] +pub(crate) struct RepositoryInspection { + #[serde(rename = "type")] + pub(crate) record_type: &'static str, + pub(crate) id: String, + pub(crate) owner: String, + pub(crate) slug: String, + pub(crate) visibility: String, + pub(crate) state: String, + pub(crate) object_format: String, + pub(crate) created_at: i64, + pub(crate) archived_at: Option, +} + +impl From for RepositoryInspection { + fn from(repository: RepositoryRecord) -> Self { + Self { + record_type: "repository", + id: repository.id, + owner: repository.owner, + slug: repository.slug, + visibility: repository.visibility, + state: repository.state, + object_format: repository.object_format, + created_at: repository.created_at, + archived_at: repository.archived_at, + } + } +} + +#[derive(Debug, Serialize)] +pub(crate) struct DumpRow { + #[serde(rename = "type")] + pub(crate) record_type: &'static str, + pub(crate) table: String, + pub(crate) columns: Vec, +} + +#[derive(Debug, Serialize)] +pub(crate) struct DumpColumn { + pub(crate) name: String, + pub(crate) value: DumpValue, +} + +#[derive(Debug, Serialize)] +#[serde(tag = "type", content = "value", rename_all = "kebab-case")] +pub(crate) enum DumpValue { + Null, + Integer(i64), + Real(String), + TextUtf8(String), + TextHex(String), + BlobHex(String), +} + +struct TableColumn { + name: String, + primary_key: i64, } #[allow( @@ -4537,6 +4794,33 @@ }) } +fn quote_identifier(identifier: &str) -> String { + format!("\"{}\"", identifier.replace('"', "\"\"")) +} + +fn dump_value(value: ValueRef<'_>) -> DumpValue { + match value { + ValueRef::Null => DumpValue::Null, + ValueRef::Integer(value) => DumpValue::Integer(value), + ValueRef::Real(value) => DumpValue::Real(format!("{value:.17e}")), + ValueRef::Text(value) => match std::str::from_utf8(value) { + Ok(value) => DumpValue::TextUtf8(value.to_owned()), + Err(_) => DumpValue::TextHex(hex(value)), + }, + ValueRef::Blob(value) => DumpValue::BlobHex(hex(value)), + } +} + +fn hex(bytes: &[u8]) -> String { + const HEX: &[u8; 16] = b"0123456789abcdef"; + let mut output = String::with_capacity(bytes.len() * 2); + for byte in bytes { + output.push(char::from(HEX[(byte >> 4) as usize])); + output.push(char::from(HEX[(byte & 0x0f) as usize])); + } + output +} + fn is_unique_constraint(error: &rusqlite::Error) -> bool { matches!( error, @@ -4552,12 +4836,7 @@ )] pub(crate) fn doctor(instance_dir: &Path) -> Result<(), StoreError> { let path = instance_dir.join(DATABASE_FILE); - let connection = Connection::open_with_flags( - path, - OpenFlags::SQLITE_OPEN_READ_WRITE | OpenFlags::SQLITE_OPEN_NO_MUTEX, - )?; - configure(&connection)?; - let store = Store { connection }; + let store = Store::open_read_only(&path)?; let actual = store.schema_version()?; if actual != SCHEMA_VERSION { return Err(StoreError::SchemaVersion { diff --git a/tests/cli.rs b/tests/cli.rs index ec0bf742f72d6cd050ec7d027722edf50ab3721a..de47307ba7bc6a616de39812842a19426f2a4017 100644 --- a/tests/cli.rs +++ b/tests/cli.rs @@ -85,6 +85,7 @@ .execute_batch(CURRENT_DATABASE) .expect("create the current database"); drop(database); + prepare_doctor_files(&instance); let output = instance.run(&[ "--config", @@ -92,7 +93,11 @@ "doctor", ]); - assert!(output.status.success()); + assert!( + output.status.success(), + "doctor failed: {}", + String::from_utf8_lossy(&output.stderr) + ); assert!(output.stdout.is_empty()); assert!(output.stderr.is_empty()); } @@ -146,6 +151,7 @@ ) .expect("create a foreign-key violation"); drop(database); + prepare_doctor_files(&instance); let output = instance.run(&[ "--config", @@ -155,7 +161,24 @@ assert_eq!(output.status.code(), Some(1)); assert!(output.stdout.is_empty()); - assert!(String::from_utf8_lossy(&output.stderr).contains("foreign key violation")); + assert!( + String::from_utf8_lossy(&output.stderr).contains("foreign key violation"), + "unexpected doctor error: {}", + String::from_utf8_lossy(&output.stderr) + ); +} + +fn prepare_doctor_files(instance: &TestInstance) { + fs::set_permissions( + instance.path().join("tit.sqlite3"), + fs::Permissions::from_mode(0o600), + ) + .expect("make the database private"); + let repositories = instance.path().join("repositories"); + fs::create_dir(&repositories).expect("create the repository directory"); + fs::set_permissions(&repositories, fs::Permissions::from_mode(0o700)) + .expect("make the repository directory private"); + create_ssh_key_fixture(&instance.path().join("ssh_host_ed25519_key")); } #[test] @@ -685,6 +708,216 @@ assert_eq!(rejected.status.code(), Some(1), "accepted {slug:?}"); assert!(rejected.stdout.is_empty()); } +} + +#[test] +fn doctor_inspect_and_dump_are_read_only_and_detect_operational_damage() { + let instance = TestInstance::new(); + create_administrator(&instance, "alice"); + let config = instance.config().to_str().expect("a UTF-8 path"); + let created = instance.run(&[ + "--config", + config, + "admin", + "repository", + "create", + "alice", + "project", + ]); + assert!(created.status.success()); + create_ssh_key_fixture(&instance.path().join("ssh_host_ed25519_key")); + + let doctor = instance.run(&["--config", config, "doctor"]); + assert!( + doctor.status.success(), + "doctor failed: {}", + String::from_utf8_lossy(&doctor.stderr) + ); + assert!(doctor.stdout.is_empty()); + + let account = instance.run(&["--config", config, "inspect", "account", "alice"]); + assert!(account.status.success()); + let account: serde_json::Value = + serde_json::from_slice(&account.stdout).expect("parse the account inspection"); + assert_eq!(account["type"], "account"); + assert_eq!(account["username"], "alice"); + assert_eq!(account["keys"].as_array().map(Vec::len), Some(1)); + + let repository = instance.run(&[ + "--config", + config, + "inspect", + "repository", + "alice", + "project", + ]); + assert!(repository.status.success()); + let repository: serde_json::Value = + serde_json::from_slice(&repository.stdout).expect("parse the repository inspection"); + assert_eq!(repository["type"], "repository"); + assert_eq!(repository["object_format"], "sha1"); + let repository_id = repository["id"] + .as_str() + .expect("read the repository identifier") + .to_owned(); + + let first_dump = instance.run(&["--config", config, "dump"]); + let second_dump = instance.run(&["--config", config, "dump"]); + assert!(first_dump.status.success()); + assert_eq!(first_dump.stdout, second_dump.stdout); + let dump_rows = String::from_utf8(first_dump.stdout).expect("read the JSON Lines dump"); + assert!(!dump_rows.is_empty()); + for line in dump_rows.lines() { + let row: serde_json::Value = serde_json::from_str(line).expect("parse a dump row"); + assert_eq!(row["type"], "sqlite-row"); + assert!(row["table"].is_string()); + assert!(row["columns"].is_array()); + } + assert!(dump_rows.contains("\"type\":\"blob-hex\"")); + + let backup_directory = TempDir::new().expect("create a backup directory"); + let backup = backup_directory.path().join("instance.tar"); + let backup_output = instance.run(&[ + "--config", + config, + "backup", + backup.to_str().expect("a UTF-8 backup path"), + ]); + assert!(backup_output.status.success()); + let checked_backup = instance.run(&[ + "--config", + config, + "doctor", + "--backup", + backup.to_str().expect("a UTF-8 backup path"), + ]); + assert!(checked_backup.status.success()); + + let database_path = instance.path().join("tit.sqlite3"); + let database = rusqlite::Connection::open(&database_path).expect("open the database"); + let repository_path = instance + .path() + .join("repositories") + .join(format!("{repository_id}.git")); + let intent_id = "11111111111111111111111111111111"; + let quarantine = repository_path + .join("objects") + .join("tit-quarantine") + .join(intent_id); + database + .execute( + "INSERT INTO git_operation_intent + (id, repository_path, actor, initial_refs, proposed_refs, event_payload, + quarantine_path, state, pack_name, created_at) + VALUES (?1, ?2, 'alice', X'', X'', X'', ?3, 'pending', NULL, 1)", + rusqlite::params![ + intent_id, + repository_path.to_str().expect("a UTF-8 repository path"), + quarantine.to_str().expect("a UTF-8 quarantine path") + ], + ) + .expect("create an incomplete intent"); + drop(database); + + let incomplete = instance.run(&["--config", config, "doctor"]); + assert_eq!(incomplete.status.code(), Some(1)); + assert!(String::from_utf8_lossy(&incomplete.stderr).contains("incomplete intents")); + let intent = instance.run(&["--config", config, "inspect", "intent", intent_id]); + assert!(intent.status.success()); + let intent: serde_json::Value = + serde_json::from_slice(&intent.stdout).expect("parse the intent inspection"); + assert_eq!(intent["state"], "pending"); + let database = rusqlite::Connection::open(&database_path).expect("reopen the database"); + assert_eq!( + database + .query_row( + "SELECT state FROM git_operation_intent WHERE id = ?1", + [intent_id], + |row| row.get::<_, String>(0), + ) + .expect("read the intent state"), + "pending" + ); + drop(database); + let repaired_intent = instance.run(&["--config", config, "repair", "intents"]); + assert!( + repaired_intent.status.success(), + "intent repair failed: {}", + String::from_utf8_lossy(&repaired_intent.stderr) + ); + let database = rusqlite::Connection::open(&database_path).expect("reopen the database"); + assert_eq!( + database + .query_row( + "SELECT state FROM git_operation_intent WHERE id = ?1", + [intent_id], + |row| row.get::<_, String>(0), + ) + .expect("read the repaired intent state"), + "abandoned" + ); + drop(database); + + fs::create_dir_all(&quarantine).expect("create quarantine debris"); + fs::write(quarantine.join("incoming.pack"), b"debris").expect("write quarantine debris"); + let debris = instance.run(&["--config", config, "doctor"]); + assert_eq!(debris.status.code(), Some(1)); + assert!(String::from_utf8_lossy(&debris.stderr).contains("quarantine debris")); + let repaired_quarantine = instance.run(&["--config", config, "repair", "quarantine"]); + assert!( + repaired_quarantine.status.success(), + "quarantine repair failed: {}", + String::from_utf8_lossy(&repaired_quarantine.stderr) + ); + + let database = rusqlite::Connection::open(&database_path).expect("reopen the database"); + database + .execute_batch("DROP INDEX audit_event_history") + .expect("remove a required index"); + drop(database); + let missing_index = instance.run(&["--config", config, "doctor"]); + assert_eq!(missing_index.status.code(), Some(1)); + assert!(String::from_utf8_lossy(&missing_index.stderr).contains("audit_event_history")); + let database = rusqlite::Connection::open(&database_path).expect("reopen the database"); + database + .execute_batch( + "CREATE INDEX audit_event_history + ON audit_event (created_at DESC, id DESC)", + ) + .expect("restore the required index"); + drop(database); + + let bad_ref = repository_path.join("refs").join("heads").join("broken"); + fs::create_dir_all(bad_ref.parent().expect("the ref has a parent")) + .expect("create the ref directory"); + fs::write(&bad_ref, b"deadbeef\n").expect("damage a Git ref"); + let damaged_git = instance.run(&["--config", config, "doctor"]); + assert_eq!(damaged_git.status.code(), Some(1)); + fs::remove_file(&bad_ref).expect("remove the damaged ref"); + + fs::set_permissions(instance.config(), fs::Permissions::from_mode(0o644)) + .expect("make the configuration unsafe"); + let unsafe_permissions = instance.run(&["--config", config, "doctor"]); + assert_eq!(unsafe_permissions.status.code(), Some(1)); + assert!(String::from_utf8_lossy(&unsafe_permissions.stderr).contains("owner-only")); + fs::set_permissions(instance.config(), fs::Permissions::from_mode(0o600)) + .expect("restore the configuration permissions"); + + let mut backup_bytes = fs::read(&backup).expect("read the backup"); + let offset = backup_bytes + .windows(b"version = 1".len()) + .position(|candidate| candidate == b"version = 1") + .expect("find archived configuration data"); + backup_bytes[offset] ^= 1; + fs::write(&backup, backup_bytes).expect("damage the backup"); + let damaged_backup = instance.run(&[ + "--config", + config, + "doctor", + "--backup", + backup.to_str().expect("a UTF-8 backup path"), + ]); + assert_eq!(damaged_backup.status.code(), Some(1)); } fn create_administrator(instance: &TestInstance, username: &str) { diff --git a/tests/support/mod.rs b/tests/support/mod.rs index fd00b5045b0dc87b1f3ac2cd9f9d3f49834b5b67..5094ec042eea2a3198d0bc7e5bc3b24d47a94d81 100644 --- a/tests/support/mod.rs +++ b/tests/support/mod.rs @@ -1,5 +1,6 @@ use std::fs; use std::net::{SocketAddr, TcpListener}; +use std::os::unix::fs::PermissionsExt; use std::path::{Path, PathBuf}; use std::process::{Command, Output}; @@ -19,6 +20,10 @@ "version = 1\npublic_url = \"https://tit.example/\"\n", ) .expect("write the configuration"); + fs::set_permissions(directory.path(), fs::Permissions::from_mode(0o700)) + .expect("make the instance directory private"); + fs::set_permissions(&config, fs::Permissions::from_mode(0o600)) + .expect("make the configuration private"); Self { directory, config } }