---
title: "Contribution guidelines and dataset format"
date: "Created: 13 September 2022; Compiled: `r BiocStyle::doc_date()`"
package: "`r BiocStyle::pkg_ver('imcdatasets')`"
author:
- name: Nicolas Damond
affiliation: [Department for Quantitative Biomedicine; University of Zurich,
Institute of Molecular Health Sciences; ETH Zurich]
email: nicolas.damond@dqbm.uzh.ch
output:
BiocStyle::html_document:
toc_float: yes
bibliography: "`r system.file('scripts', 'ref.bib', package='imcdatasets')`"
vignette: >
%\VignetteIndexEntry{"Contributing guidelines and datasets format"}
%\VignetteEngine{knitr::rmarkdown}
%\VignetteEncoding{UTF-8}
---
```{r style, echo=FALSE}
knitr::opts_chunk$set(error=FALSE, warning=FALSE, message=FALSE)
library(BiocStyle)
```
# **Introduction**
This vignette describes the procedure to contribute new datasets to the
`imcdatasets` package and contains guidelines for dataset formatting.
# **Contribution guidelines**
Contributions or suggestions for new imaging mass cytometry (IMC) datasets to
add to the `imcdatasets` package are always welcome. New datasets can be
suggested by opening an issue at the `imcdatasets`
[GitHub page](https://github.com/BodenmillerGroup/imcdatasets/issues). The only
requirements are that the new dataset *(i)* is publicly available and *(ii)*
has been described in a published scientific article.
Details about creating Bioconductor's `ExperimentHub` packages
[are available here](https://bioconductor.org/packages/HubPub).
## Create a data generation script
The first step is to create a new branch at the `imcdatasets`
[GitHub page](https://github.com/BodenmillerGroup/imcdatasets/branches).
Then, create an R markdown (`.Rmd`) script in `.inst/scripts/` to generate the
data objects:
* Download single cell data, multiplexed images and cell segmentation masks.
* Format the single cell data into a
[SingleCellExperiment](https://bioconductor.org/packages/SingleCellExperiment)
object.
* Format the images and masks into
[CytoImageList](https://bioconductor.org/packages/CytoImageList) objects.
* Save the three data objects so that they can be uploaded to `ExperimentHub`.
* The three data objects must contain matching information so that they can be
associated by the [cytomapper](https://bioconductor.org/packages/cytomapper)
package, as described in the [imcdataset vignette](https://www.bioconductor.org/packages/release/data/experiment/vignettes/imcdatasets/inst/doc/imcdatasets.html#5_Usage).
The `.Rmd` script must be formatted in the same way as pre-existing scripts.
Examples can be found [here](https://github.com/BodenmillerGroup/imcdatasets/blob/master/inst/scripts/make-Damond-2019-Pancreas.Rmd) and [here](https://github.com/BodenmillerGroup/imcdatasets/blob/master/inst/scripts/make-JacksonFischer-2020-BreastCancer.Rmd). Each step should be clearly and comprehensively
documented.
**For usability of the package and consistency across datasets, the data **
**objects must be formatted as described in the** `Dataset format` **section **
**below.**
## Update the documentation
Other files in the `imcdatasets` package should be updated to include the new
dataset:
* Make a new `./R/Lastname_Year_Type.R` file with a function to load the new
dataset and extensive documentation. Examples can be found [here](https://github.com/BodenmillerGroup/imcdatasets/blob/master/R/Damond_2019_Pancreas.R)
and [here](https://github.com/BodenmillerGroup/imcdatasets/blob/master/R/JacksonFischer_2020_BreastCancer.R).
* Run roxygenize to generate `man` documentation files (go to the `imcdatasets`
directory and run `roxygen2::roxygenize(".")`).
* Update the `./inst/scripts/make-metadata.R` R script and run it. This will
update the `./inst/extdata/metadata.csv` file that is used by `ExperimentHub`
to provide metadata information about datasets.
* Add the reference of the paper that describes the dataset to the
`./inst/scripts/ref.bib` file.
* Add the new dataset to the `./inst/extdata/alldatasets.csv` file.
* Add the new dataset to the dataset list in `./tests/testthat/test_loading.R`.
## Open a pull request
After these steps have been completed, open a pull request at the
[imcdataset GitHub page](https://github.com/BodenmillerGroup/imcdatasets/pulls).
The package maintainers will do the following:
* Check the R markdown script for data generation.
* Generate the data objects by knitting the R markdown script.
* Make sure the data objects are well formatted and consistent with the other
datasets.
* Check all the new package metadata and documentation.
* Upload the data objects to `AWS S3` and announce the upload to Bioconductor
Hubs.
* Download the data objects from `ExperimentHub` and check the format again.
* Update the `NEWS`, `DESCRIPTION` (add new contributor, version bump) and
`README.md` (if needed) files.
* Build and check the package, make sure it passes all R and Bioconductor
checks.
* Push to GitHub and check that the `imcdatasets` package can be installed
from there.
* Test the package functionality in R.
* Once everything works, approve the pull request and merge with the master
branch.
Contributors will be recognized by having their names added to the DESCRIPTION
file of the `imcdatasets` package.
# **Dataset format**
The `imcdatasets` package is meant to provide quick and easy access to
published and curated IMC datasets. Each dataset consists of three data objects
that can be retrieved individually:
* __Single cell data__ in the form of a [SingleCellExperiment](https://bioconductor.org/packages/SingleCellExperiment)
object.
* __Multichannel images__ formatted into a [CytoImageList](https://bioconductor.org/packages/CytoImageList) object.
* __Cell segmentation masks__ formatted into a [CytoImageList](https://bioconductor.org/packages/CytoImageList) object.
The three data objects can be mapped using unique `image_name` values contained
in the metadata of each object.
For consistency across datasets, the guidelines below must be followed when
creating a new dataset.
## Single cell data
Single cell data should be formatted into a [SingleCellExperiment](https://bioconductor.org/packages/SingleCellExperiment)
object named `sce` that contains the following slots:
- `colData`: observations metadata.
- `rowData`: marker metadata.
- `assays`: marker expression levels per cell.
- `colPairs` *(optional)*: neighborhood information.
### colData
The `colData` entry of the `SingleCellExperiment` object is a `DataFrame` that
contains observation metadata; i.e., cells, slides, tissue, patients, .... It
is recommended that all column names have a prefix that indicates the level of
observation (e.g. `cell_`, `slide_` , `tissue_`, `patient_`, `tumor_`).
The following columns are required:
- `image_name` and/or `image_number`: unique image (ablated ROI) name,
respectively number. Should map to the `image_name`/`image_number` column(s)
in the metadata of the `images` and `masks` objects.
- `cell_number`: integer representing cell numbers. Should map to the values of
cell segmentation masks.
- `cell_id`: a unique cell identifier defined as
{`image_number` `_` `cell_number`} (e.g., `7_138`).
- `cell_x` and `cell_y`: position of the cell centroid on the image. These
columns are used as `SpatialCoords` when converting to a [SpatialExperiment](https://bioconductor.org/packages/SpatialExperiment)
object.
In addition, `colnames(sce)` should be set as `colData(sce)$cell_id`.
### rowData
The `rowData` entry of the `SingleCellExperiment` is a `DataFrame` that
contains marker (protein, RNA, probe) information.
The following columns are required in the `rowData` entry:
- `channel`: a unique integer that maps to the channels of the associated
multichannel images.
- `metal`: the metal isotope used for detection, formatted as
{ `ChemicalSymbol` `IsotopeMass`} (e.g., `Y89`, `In115`, `Yb176`, `Bi209`).
- `name`: marker name used in the publication that describes the dataset.
- `full_name`: full marker name.
- `short_name`: abbreviated marker name, preferably following the official [UniProt](https://www.uniprot.org) nomenclature.
For the `full_name` and `short_name` columns, the following guidelines apply:
* In `short_name`, all dashes, dots and spaces should removed or replaced with
underscores.
* For post-translationally modified proteins:
* Prefix the `full_name` with the modification type (e.g., `phospho-`,
`methyl-`) and suffix it with the modified aminoacids (e.g., `[S28]`).
* Prefix the `short_name` with an abbreviation of the modification type
(e.g., `p_`, `me_`) and do not indicate the modified aminoacids, unless there
is a possible confusion with another target in the dataset.
```{r echo=FALSE, results='hide'}
marker_names <- data.frame(
full_name = c(
"Carbonic anhydrase IX",
"CD3 epsilon",
"CD8 alpha",
"E-Cadherin",
"cleaved-Caspase3 + cleaved-PARP",
"Cytokeratin 5",
"Forkhead box P3",
"Glucose transporter 1",
"Histone H3",
"phospho-Histone H3 [S28]",
"Ki-67",
"Myeloperoxidase",
"Programmed cell death protein 1",
"Programmed death-ligand 1",
"phospho-Rb [S807/S811]",
"Smooth muscle actin",
"Vimentin",
"Iridium 191",
"Iridium 193"
),
short_name = c(
"CA9",
"CD3e",
"CD8a",
"CDH1",
"cCASP3_cPARP",
"KRT5",
"FOXP3",
"SLC2A1",
"H3",
"p_H3",
"Ki67",
"MPO",
"PD_1",
"PD_L1",
"p_Rb",
"SMA",
"VIM",
"DNA1",
"DNA2"
)
)
```
```{r echo=FALSE, results='asis'}
knitr::kable(
marker_names,
caption = "'full_name' and 'short_name' examples for some commonly
used markers"
)
```
In addition, `rownames(sce)` should be set as `rowData(sce)$short_name`.
### assays
The `assays` slot of the `SingleCellExperiment` contains counts matrices
representing marker expression levels per cell and channel.
It should at least contain a `counts` matrix with raw ion counts. The `assays`
slot can also contain additional matrices with commonly used counts
transformations, or counts transformations that were used in the publication
that describes the dataset. All counts transformations must be documented in
the `.R` function used to load the dataset. Common examples include:
* `exprs`: asinh-transformed counts. For IMC, a cofactor of 1 is typically
used.
* `quant_norm`: counts censored (e.g., at the 99th percentile) and scaled from
0 to 1.
### colPairs
Neighborhood information, such as a list of cells that are localized next to
each other, can be stored as a [SelfHits](https://bioconductor.org/packages/devel/bioc/vignettes/SingleCellExperiment/inst/doc/intro.html#6_Storing_row_or_column_pairings) object in the `colPair` slot of the `SingleCellExperiment` object.
## Images and masks
### Images
Multichannel images are stored in a [CytoImageList](https://bioconductor.org/packages/CytoImageList) object named
`images`.
Channel names of the `images` object (`channelNames(images)`) must map to
`rownames(sce)` (marker short names).
The metadata slot (`mcols(images)`) must contain an `image_name` column that
maps to the `image_name` column of `colData(sce)`, and to the `image_name` column of `mcols(masks)`. This information is used by [cytomapper](https://bioconductor.org/packages/cytomapper)
to associate multichannel images, cell segmentation masks, and single cell data.
### Masks
Cell segmentation masks are stored in a [CytoImageList](https://bioconductor.org/packages/CytoImageList) object named
`masks`.
The values of the masks should be integers mapping to the `cell_number` column
of `colData(sce)`. This information is used by [cytomapper](https://bioconductor.org/packages/cytomapper) to associate single
cell data and cell segmentation masks.
The metadata slot (`mcols(masks)`) must contain an `image_name` column that
maps to the `image_name` column of `colData(sce)`, and to the `image_name` column of `mcols(images)`. This information is used by [cytomapper](https://bioconductor.org/packages/cytomapper) to associate
multichannel images, cell segmentation masks, and single cell data.
# **Session info** {.unnumbered}
```{r sessionInfo, echo=FALSE}
sessionInfo()
```