Symlink Tool — Technical Reference

This vignette assumes you’ve read the Quick Start guide and covers:

  1. The Symlink Composer (SLC) for cross-project queries
  2. Multiple roots per tool instance
  3. Custom symlink types via the plugin architecture
  4. Reports and discrepancy detection

Each section is self-contained — run them independently or in order.

2 — Multiple Roots

A single SLT instance can manage several root directories at once. This is useful when one pipeline run produces outputs that land in different locations — e.g. pre-model inputs and post-model results — but you want them versioned in lockstep under the same version_name.

Setup

base <- file.path(tempdir(), "slt_tech", "multi_root")

root_input  <- file.path(base, "to_model")
root_output <- file.path(base, "modeled")
dir.create(root_input,  recursive = TRUE, showWarnings = FALSE)
dir.create(root_output, recursive = TRUE, showWarnings = FALSE)

slt <- SLT$new(
  root_list = list(
    input  = root_input,
    output = root_output
  ),
  central_log_root = base
)

Two roots, one tool. Every mark or make_new_version_folder call operates on both roots in parallel.

Create and mark

v1 <- next_version(root_input, "2024_06_01")
slt$make_new_version_folder(version_name = v1)

# Populate with dummy files
invisible(file.create(file.path(root_input,  v1, "prep.csv")))
invisible(file.create(file.path(root_output, v1, "results.csv")))

v2 <- next_version(root_input, "2024_06_01")
slt$make_new_version_folder(version_name = v2)
invisible(file.create(file.path(root_input,  v2, "prep.csv")))
invisible(file.create(file.path(root_output, v2, "results.csv")))

slt$mark_best(version_name = v1, user_entry = list(comment = "Initial run"))
print_tree(base, level = 2)
#> |-- log_symlinks_central.csv
#> |-- modeled
#> |  |-- 2024_06_01.01
#> |  |-- 2024_06_01.02
#> |  |-- best -> /tmp/RtmpGQGgAP/slt_tech/multi_root/modeled/2024_06_01.01
#> |  `-- report_key_versions.csv
#> `-- to_model
#>    |-- 2024_06_01.01
#>    |-- 2024_06_01.02
#>    |-- best -> /tmp/RtmpGQGgAP/slt_tech/multi_root/to_model/2024_06_01.01
#>    `-- report_key_versions.csv

The best symlink appears in both roots, pointing to v1 in each. Promoting v2 demotes v1 everywhere at once.

suppressMessages(
  slt$mark_best(version_name = v2, user_entry = list(comment = "Revised covariates"))
)
print_tree(base, level = 1)
#> |-- log_symlinks_central.csv
#> |-- modeled
#> `-- to_model

4 — Reports and Discrepancy Detection

SLT auto-generates a report_key_versions.csv on every mark call. Additional reports are available via $make_reports().

Setup

base_rpt <- file.path(tempdir(), "slt_tech", "reports")
dir.create(base_rpt, recursive = TRUE, showWarnings = FALSE)

slt_rpt <- SLT$new(
  root_list        = list(output = base_rpt),
  central_log_root = base_rpt
)
for (i in 1:4) {
  ver <- next_version(base_rpt, "2024_11_01")
  slt_rpt$make_new_version_folder(version_name = ver)
}

suppressMessages({
  slt_rpt$mark_best(version_name = "2024_11_01.03",
                    user_entry = list(comment = "Production run"))
  slt_rpt$mark_keep(version_name = "2024_11_01.01",
                    user_entry = list(comment = "Baseline comparison"))
})

Run all reports

suppressMessages(slt_rpt$make_reports())
print_tree(base_rpt, level = 1)
#> |-- 2024_11_01.01
#> |-- 2024_11_01.02
#> |-- 2024_11_01.03
#> |-- 2024_11_01.04
#> |-- best -> /tmp/RtmpGQGgAP/slt_tech/reports/2024_11_01.03
#> |-- keep_2024_11_01.01 -> /tmp/RtmpGQGgAP/slt_tech/reports/2024_11_01.01
#> |-- log_symlinks_central.csv
#> |-- report_all_logs_non_symlink.csv
#> |-- report_all_logs_symlink.csv
#> |-- report_all_logs.csv
#> `-- report_key_versions.csv

Key-version report

Current state of every symlink-marked folder.

  • Runs automatically when marking/unmarking
  • Only tracks symlinks defined in the package registry
data.table::fread(file.path(base_rpt, "report_key_versions.csv"))[, ..show_vars]
#>    log_id         timestamp  version_name       action             comment
#>     <int>            <char>        <char>       <char>              <char>
#> 1:      1 2026_11_13_134943 2024_11_01.03 promote_best      Production run
#> 2:      1 2026_11_13_134943 2024_11_01.01 promote_keep Baseline comparison

Non-symlinked folders

Folders that exist but have no active symlink — useful for identifying cleanup candidates.

data.table::fread(file.path(base_rpt, "report_all_logs_non_symlink.csv"))[, ..show_vars]
#>    log_id         timestamp  version_name action     comment
#>     <int>            <char>        <char> <char>      <char>
#> 1:      0 2026_11_13_134943 2024_11_01.02 create log created
#> 2:      0 2026_11_13_134943 2024_11_01.04 create log created

Discrepancy report

If any versioned logs have unexpected structure (missing fields, extra rows, schema drift), a REPORT_DISCREPANCIES.csv file is written. A clean project produces no discrepancy file.

  • Runs automatically when marking/unmarking
try(data.table::fread(file.path(base_rpt, "REPORT_DISCREPANCIES.csv")))
#> Error in data.table::fread(file.path(base_rpt, "REPORT_DISCREPANCIES.csv")) : 
#>   File '/tmp/RtmpGQGgAP/slt_tech/reports/REPORT_DISCREPANCIES.csv' does not exist or is non-readable. getwd()=='/tmp/RtmpOvpDLq/Rbuildfac415a535a/vmTools/vignettes'

Cleanup