Used to evaluate Life Sciences research software, OpenEBench is an observatory for software quality based on the automated monitoring of FAIR for research software metrics and indicators. The OpenEBench platform supports both the technical monitoring of scientific software and scientific benchmarking activities carried out by Life Sciences Communities. Its architecture has three different engagement levels that allow communities at different maturity stages to make use of the platform. It also connects with ELIXIR Core Data Resources and Deposition databases to use data needed by the Scientific Communities activities. The expected impact of the integration with EOSC services is that Life Science researchers will have semantically annotated, up-to-date collections of benchmarked analytical workflows and tools, organised by scientific communities for specific topics, which can be deployed across heterogeneous systems.
OpenEBench as platform has the overall objectives:
- Provide guidance and infrastructure support for community-led scientific benchmarking efforts.
- Provide an observatory for software quality based on the automated monitoring of FAIR for research software metrics and indicators.
- Work towards the sustainability of the platform by adopting, integrating and promoting principles on Open Software, Open Data and Open Science.
- Adopt community-led standards, protocols and/or including the Global Alliance for Genomics and Health (GA4GH), ELIXIR and the European Open Science Cloud (EOSC).
OpenEBench can engage with different end-user profiles across the Life Sciences communities and beyond.
- Developers, who have a reference place to identify current challenges and relevant data sets for developing new algorithms and/or measure the impact of new developments. OpenEBench offers the possibility to developers to compare the scientific performance of their solutions with others from the community. Ultimately, it helps to improve their methods and disseminate their results thanks to publications and results spreading.
- OpenEBench assists Communities in the organization of their scientific benchmarking activities and the identification of new trends in their concrete area by providing examples of assessment metrics already in use in other communities, contributing to results dissemination and establishing good practices.
- Researchers mainly benefit from getting guidance about choosing the best resource for their research needs and be aware of the latest advancements in the area by getting information from trusted experts and staying up to date with new developments.
- Funders are able to maximize impact from projects which include the development of new software resources and/or improve the existing ones.