# 02_QUEST_FIG3 SPEC

## Instructions:
You will EDIT this SPEC so your agent will analyze real single-cell data and create 3+ figures (replacing the mock data and placeholder figures generated in 01_QUEST_START_HERE).  
You should also write a series of tests to:
- verify the code works
- validate the code's results reflect the underlying biology

In this document, over write anything in {{}} to get Gemini to work with you. 

(We have filled in other sections to save you time and focus your attention on the more valuable educational aspects.)

This is a suggested SPEC structure, but feel free to add or change content.


---

## 1. GOALs and Background 

**Objective:** Start a single cell analysis with SCimilarity foundation model by creating cell *embeddings* and *labels* from new scRNA data. **Reproduce published results: Scimilarity paper Figure 3** comparing cell *labels* from SCimilarity predictions against held-out, author-annotated results to assess model processing and output. 

### Figure 3 description: 
- **3b (Author Annotations):** A UMAP plot of the kidney dataset where cells are colored by the original author-assigned labels (ground truth).
- **3c (SCimilarity Predictions):** The same UMAP plot as 3b, but cells are colored by labels automatically predicted by the SCimilarity foundation model.
- **3d (Concordance Heatmap):** A square, diagonalized heatmap matrix comparing harmonized author cell types vs. SCimilarity predicted cell types. The x-axis and y-axis must be perfectly aligned and matched to each other, displaying the exact same cell type labels to form a clear diagonal line of agreement.


### Deliverables:
1. Python scripts that extract author annotations and calculate Scimilarity predictions from a real data set
2. Tests that verify and validate the output data

## 2. IMPLEMENTATION

### BIOLOGY
#### **Cell label terminology** might need standardizing
   - Need a way to compare Scimilarity model output, Fig 3 annotations, and author annotations, in a harmonized way based on the paper.
   - **Methodology (Tiered Approach with Ontology):**
     1. **String Normalization:** Clean and standardize both author and model labels (e.g., lowercase, remove special characters).
     2. **Direct Mapping:** Utilize a predefined dictionary to map known synonyms specific to the kidney dataset.
     3. **Highest-Probability Validation:** Use the concordance matrix to identify relationships between unmapped labels.
     4. **Ontology Integration (FAIR Data):** Map all successfully harmonized cell types to standard **Cell Ontology (CL)** identifiers (e.g., CL:0000653 for podocyte) to ensure interoperability.
   - **Appropriateness & Validation:** A mapping is considered appropriate if there is at least an 80% overlap (concordance) between the author's label cluster and the SCimilarity prediction. Mappings that fall below this threshold will be flagged for manual review. Outputs will include a `label_mapping.csv` containing the Author Label, SCimilarity Label, Harmonized Label, and CL ID for transparency.


### Tech stack and project structure.

#### Languages
- Use python for most processing.

#### Architecture
- Use the existing project structure as outlined in 01_QUEST_START_HERE.md.

#### Interactive plot infrastracture
1. **Local File Serving:** 
   - *Gotcha:* Standard Python `http.server` often fails to resolve symlinked data directories outside its root, leading to 404s.
   - *Fix:* Physically copy the JSON files: `mkdir -p web/data && cp results/data_subsample/* web/data/` before serving.
2. **Plotly Performance:**
   - *Gotcha:* 10,000 points using standard `scatter` SVG rendering is slow.
   - *Fix:* Use `type: 'scattergl'` for WebGL hardware acceleration.
3. **Interactive Highlighting:**
   - *Gotcha:* Updating the data arrays for every dropdown change is too slow.
   - *Fix:* Group cells into separate Plotly traces by cell type. To highlight, change `marker.opacity` to `0.05` for non-matching traces and keep it at full opacity for the matching trace.
4. Do not invent new file formats for biological data. Use standard formats where possible. 



## 3. INPUTS

**Data Sources:**
- **SCimilarity Model:** Local path `/data/models/model_v1.1` (Symlinked to `models/model_v1.1` in the workspace).
- **Kidney Dataset:** Kretzler/KPMP kidney dataset is used in Figure 3.
  - Source: `gs://rb-wkshp-bioit26-data/data/kretzler_kidney.h5ad`
  - Destination: `data/kretzler_kidney.h5ad`

Use the existing project structure as outlined in 01_QUEST_START_HERE.md.

Use the existing python virtualenv installed in this directory.

## 4. OUTPUTS

Use the json data in web/data/ to understand the structure of the output data. Document that here.
- `umap_data.json`: Contains `x_author`, `y_author`, `x_scim`, `y_scim`, `author_label_idx`, and `pred_label_idx` for the scatter plots.
- `concordance_matrix.json`: Contains the `values`, `author_labels`, and `pred_labels` for the heatmap.
- `metadata.json`: Contains the `cell_types` list mapping indices to label strings.

### TABLEs
- Write processing results to files (csv or appropriate format) for additional testing/inspection.
- **Additional Outputs:** 
  - `label_mapping.csv`: The mapping table created during the harmonization process.
  - `annotated_predictions.h5ad`: The full AnnData object with the newly appended SCimilarity predictions and UMAP coordinates, saved for downstream re-use.

### Figure 3 test and interpretation: 
- **Interpretation:** A successful execution should result in a strongly diagonal concordance heatmap, indicating that the SCimilarity model's unsupervised predictions closely mirror the author's manual annotations for the majority of distinct cell types.


## 5. TESTS

### VALIDATION Tests (Accuracy & Science)

- **Biological Accuracy:** We will calculate the mean concordance across all mapped cell types. We expect an average concordance score of at least 80% to validate that the model is functioning as described in Figure 3.

- **Automated Tests & Metrics:**
  - Verify that the resulting UMAP coordinates do not contain `NaN` or `Infinity` values.
  - Verify that the number of cells in the input `.h5ad` matches the number of cells in the output JSON files.
  - Calculate and print a summary metric: "Percentage of cells correctly mapped according to the tiered harmonization strategy."

### VERIFICATION (Correct execution)

- Write a small text file summarizing the input data, including structure, dimensions and how author data can be used for our task. 

#### Unit tests (working, reproducible code)

Add a suite of unittests in tests/unit/ that run with the standard
library unittest package. These tests should aim for >75% test coverage
of the python files in src/.

- All APIs should be mocked
- Create a set of mock data from a file in data/ that is structurally identical but much smaller that the real data to aid with testing
- Write a testing README with an overview of the testing strategy and coverage
- Check that the tests pass
- Write a series of test that ensures that the server in web/ can be run without errors, including javascript errors.




--- 
copyright: © 2026 Sonia Timberlake & Ryan Bellmore
license: Proprietary - Authorized Workshop Participants Only
distribution_allowed: false