Publication of distribution on DaRUS

For every new (major) distribution release, we publish an archive of all included sources on DaRUS. As an example, see the v2104.0 data set.

Authorship

Authors of the data set are:

  • All current maintainers
  • Everybody who significantly contributed to the delta between the last and this major distribution release. Significant contributions need to go beyond good first issues. Actively maintaining a component or significant reviewing does also qualify as significant contribution.
  • Supervisors who actively contributed ideas to the delta between the last and this major distribution release

Order of authors: maintainers are listed first followed by everybody else, both in alphabetical order.

Automation

  1. Install EasyDataverse

    As of March 2024, a specific branch was required:

    pip install git+https://github.com/gdcc/easyDataverse@flexible-connect

    These new required features are, however, scheduled to be released. Likely, installation from PyPI is possible in the future.

  2. Download previous distribution

    from easyDataverse import Dataverse

SERVER_URL = "https://darus.uni-stuttgart.de"

# get the token from DaRUS after logging in; valid for one year
API_TOKEN = "yourToken"

# DOI of preCICE Distribution Version v2211.0
PID = "doi:10.18419/darus-3576"

darus = Dataverse(SERVER_URL, API_TOKEN)
# pull data and metadata, data is written into subfolder old_files, which is also created
dataset = darus.load_dataset(PID, filedir="old_files")

# write metadata into a json file
with open("dataset.json", "w") as f:
  f.write(dataset.json())

  3. Edit metadata manually

    Copy dataset.json to dataset_new.json and update manually. Keep the old dataset_id.

  4. Download components

    #!/bin/bash

repo_names=()
is_releases=() # whether a component is a release (true) or a commit (false)
versions=() # for a release, this is the version number (without "v"); for a commit, this is the hash (of length  7)

# TODO: update all version; current example reflects distribution v2404.0

# preCICE
repo_names+=("precice"); is_releases+=(true); versions+=("3.1.1")

# Tools
repo_names+=("aste"); is_releases+=(true); versions+=("3.1.0")
repo_names+=("ci-images"); is_releases+=(false); versions+=("b421a49")
repo_names+=("config-visualizer"); is_releases+=(true); versions+=("1.1.3")
repo_names+=("config-visualizer-gui"); is_releases+=(true); versions+=("0.1.0")
repo_names+=("fmi-runner"); is_releases+=(true); versions+=("0.2.1")
repo_names+=("micro-manager"); is_releases+=(true); versions+=("0.4.0")

# Bindings
repo_names+=("fortran-module"); is_releases+=(false); versions+=("dc88c3b")
repo_names+=("PreCICE.jl"); is_releases+=(true); versions+=("3.1.0")
repo_names+=("matlab-bindings"); is_releases+=(true); versions+=("3.1.0")
repo_names+=("python-bindings"); is_releases+=(true); versions+=("3.1.0")
repo_names+=("rust-bindings"); is_releases+=(true); versions+=("3.1.0")

# Adapters
repo_names+=("calculix-adapter"); is_releases+=(true); versions+=("2.20.1")
repo_names+=("code_aster-adapter"); is_releases+=(false); versions+=("b797fcc")
repo_names+=("dealii-adapter"); is_releases+=(false); versions+=("4c6d092")
repo_names+=("dune-adapter"); is_releases+=(false); versions+=("75edcc3")
repo_names+=("dumux-adapter"); is_releases+=(true); versions+=("2.0.0")
repo_names+=("fenics-adapter"); is_releases+=(true); versions+=("2.1.0")
repo_names+=("openfoam-adapter"); is_releases+=(true); versions+=("1.3.0")
repo_names+=("su2-adapter"); is_releases+=(false); versions+=("64d4aff")

# Tutorials
repo_names+=("tutorials"); is_releases+=(true); versions+=("202404.0")

# VM
repo_names+=("vm"); is_releases+=(true); versions+=("202404.0.0")

# Website and documentation
repo_names+=("precice.github.io"); is_releases+=(true); versions+=("202404.0.0")

download_release () {
  repo_name=$1
  version=$2
  wget -nv -O ${repo_name}-${version}.tar.gz https://github.com/precice/${repo_name}/archive/refs/tags/v${version}.tar.gz
}

download_commit () {
  repo_name=$1
  hash=$2

  wget -nv https://github.com/precice/${repo_name}/archive/${hash}.zip
  unzip -q ${hash}.zip
  rm ${hash}.zip
  mv ${repo_name}-${hash}* ${repo_name}-${hash}
  tar -czf ${repo_name}-${hash}.tar.gz ${repo_name}-${hash}
  rm -r ${repo_name}-${hash}
}

# prepare separate folder for new components
rm -rf new_files
mkdir new_files
cd new_files

for ((i=0; i<${#repo_names[@]}; i++)); do
    repo_name="${repo_names[$i]}"
    is_release="${is_releases[$i]}"
    version="${versions[$i]}"

    if [ "$is_release" == "true" ]; then
        download_release "$repo_name" "$version"
    else
        download_commit "$repo_name" "$version"
    fi
done

    Once there is a machine-readable distribution, simplify the bash script accordingly.

  5. Upload to DaRUS

    from easyDataverse import Dataverse

SERVER_URL = "https://darus.uni-stuttgart.de"

# get the token from DaRUS after logging in; valid for one year
API_TOKEN = "yourToken"

darus = Dataverse(SERVER_URL, API_TOKEN)

dataset = darus.dataset_from_json(open("./dataset_new.json"))

# datasetContactEmail is a field that cannot yet be downloaded, need to provide manually
dataset.citation.dataset_contact[0].email = "benjamin.uekermann@ipvs.uni-stuttgart.de"

dataset.add_directory(
  dirpath="./new_files/",
)

# delete pid to upload to new dataset, otherwise existing one overwritten
dataset.p_id = None
# upload to dataverse of US3
dataset.upload("ipvs_us3")

  6. Review and publish

    The upload returns a url, which gives you access to the dataset. Review carefully. Currently, for example, licenses still need manual editing. Also the PDF version of the docs, a README.md, and descriptions of all components are missing. Once ready, publish (i.e. send to DaRUS team for review).