opensearch-project/opensearch-benchmark-workloads

Add Integration Tests and CI Actions to Prevent Bugs [FEATURE]

Open

#71 opened on Mar 24, 2023

 (1 comment) (0 reactions) (0 assignees)Python (137 forks)auto 404
enhancementgood first issue

Repository metrics

Stars
 (33 stars)
PR merge metrics
 (PR metrics pending)

Description

Is your feature request related to a problem?

There are no CI tests being run on PRs. It'd be nice to have a way for us to quickly run "integration tests" to changes made to workloads does not introduce errors. This has happened a few times already when users updated so and nyc_taxis, which affected other users using OSB (see example issue here). This would reduce the turn-around-time required for maintainers to dive into the code base to understand where the bug was introduced.

What solution would you like?

Utilize Github Actions to run the following:

  • Run a "lint" on formatting of all official workload's files
  • Run an OSB command with --test-mode appended for the official workload that's having the proposed change

What alternatives have you considered?

  • Using docker image but the issue there is that it would be using the changes already incorporated in official OSB workloads. However, we could get around this by pointing it to the user forked workload repository in benchmark.ini
[workloads]
default.url = https://github.com/opensearch-project/opensearch-benchmark-workloads

Do you have any additional context?

Add any other context or screenshots about the feature request here.

Contributor guide