# 02_QUEST_FIG3 SPEC ## Instructions: You will EDIT this SPEC so your agent will analyze real single-cell data and create 3+ figures (replacing the mock data and placeholder figures generated in 01_QUEST_START_HERE). You should also write a series of tests to: - verify the code works - validate the code's results reflect the underlying biology In this document, over write anything in {{}} to get Gemini to work with you. (We have filled in other sections to save you time and focus your attention on the more valuable educational aspects.) This is a suggested SPEC structure, but feel free to add or change content. --- ## 1. GOALs and Background **Objective:** Start a single cell analysis with SCimilarity foundation model by creating cell *embeddings* and *labels* from new scRNA data. **Reproduce published results: Scimilarity paper Figure 3** comparing cell *labels* from SCimilarity predictions against held-out, author-annotated results to assess model processing and output. ### Figure 3 description: - **3b (Author Annotations):** UMAP of cells, colored by the *author's* annotation. - **3c (SCimilarity Predictions):** The same cells, colored by the model's *predicted* cell type. - **3d (Concordance Heatmap):** A concordance heatmap. Rows = predicted types, columns = author-annotated types. ### Deliverables: 1. Python scripts that extract author annotations and calculate Scimilarity predictions from a real data set 2. Tests that verify and validate the output data ## 2. IMPLEMENTATION ### BIOLOGY #### **Cell label terminology** might need standardizing - Need a way to compare Scimilarity model output, Fig 3 annotations, and author annotations, in a harmonized way based on the paper. - Gemini should suggest 4 ways to reimplement cell label terminology and then have the user pick - Standardized Ontology Mapping (The Rigorous Science Approach): Map both sets of labels to a Cell Ontology (CL). We would find the CL ID for every author label, every fig 3 annotation, and every SCimilarity prediction. ### Tech stack and project structure. #### Languages - Use python for most processing #### Architecture - Use the existing project structure as outlined in 01_QUEST_START_HERE.md. #### Interactive plot infrastracture 1. **Local File Serving:** - *Gotcha:* Standard Python `http.server` often fails to resolve symlinked data directories outside its root, leading to 404s. - *Fix:* Physically copy the JSON files: `mkdir -p web/data && cp results/data_subsample/* web/data/` before serving. 2. **Plotly Performance:** - *Gotcha:* 10,000 points using standard `scatter` SVG rendering is slow. - *Fix:* Use `type: 'scattergl'` for WebGL hardware acceleration. 3. **Interactive Highlighting:** - *Gotcha:* Updating the data arrays for every dropdown change is too slow. - *Fix:* Group cells into separate Plotly traces by cell type. To highlight, change `marker.opacity` to `0.05` for non-matching traces and keep it at full opacity for the matching trace. 4. Do not invent new file formats for biological data. Use standard formats where possible. ## 3. INPUTS **Data Sources:** - **SCimilarity Model:** Local path `/data/models/model_v1.1` (Symlinked to `models/model_v1.1` in the workspace). - **Kidney Dataset:** Kretzler/KPMP kidney dataset is used in Figure 3. - Source: `gs://rb-wkshp-bioit26-data/data/kretzler_kidney.h5ad` - Destination: `data/kretzler_kidney.h5ad` Use the existing project structure as outlined in 01_QUEST_START_HERE.md. Use the existing python virtualenv installed in this directory. ## 4. OUTPUTS Use the json data in web/data/ to understand the structure of the output data. Document that here. ### TABLEs - Write processing results to files (csv or appropriate format) for additional testing/inspection. - Tables should have 6 columns. 1: UMAP 1, 2: UMAP 2, 3: Original model label before Cell Ontology (CL) applied, 4: Model label after cell ontology (CL) applied, 5: Original author label before Cell Ontology (CL) applied, 6: Author label after cell ontology (CL) applied ### Figure 3 test and interpretation: Gemini should interpret both Figure 3 inputs and visualized outputs. Outputs figures should be compared by machine vision. ## 5. TESTS ### VALIDATION Tests (Accuracy & Science) - The set of labels for 1. author labels after applying the Cell Ontology (CL) and 2. SCimilarity labels after applying the Cell Ontology (CL) should be identical. These are columns 4 and 6 in the output table. - The metric should be a similarity or correlation score between the two sets of labels. 100% is the target. Output the score. - The labels before and after applying Cell Ontology (CL) for both 1: author labels and 2: SCimilarity labels should describe the same cell type. - The metric should be a similarity score between the two sets of labels (before and after CL). 90% is the target. Output the score. - the output figure 3b using author label and output figure 3c using scimilarity labels should have the same shape, the only difference should be the color. - Convert to black and white and compare the figures pixel by pixel. The pixels should be 100% identical between figures 3b and 3c. Output the score. ### VERIFICATION (Correct execution) - Write a small text file summarizing the input data, including structure, dimensions and how author data can be used for our task. #### Unit tests (working, reproducible code) Add a suite of unittests in tests/unit/ that run with the standard library unittest package. These tests should aim for >75% test coverage of the python files in src/. - All APIs should be mocked - Create a set of mock data from a file in data/ that is structurally identical but much smaller that the real data to aid with testing - Write a testing README with an overview of the testing strategy and coverage - Check that the tests pass - Write a series of test that ensures that the server in web/ can be run without errors, including javascript errors. --- copyright: © 2026 Sonia Timberlake & Ryan Bellmore license: Proprietary - Authorized Workshop Participants Only distribution_allowed: false