vectordotdev/vector

MessagePack codec (or VRL functions)

Open

#8,675 opened on Aug 12, 2021

View on GitHub
 (2 comments) (8 reactions) (0 assignees)Rust (2,126 forks)batch import
domain: codecsdomain: vrlgood first issuetype: featurevrl: stdlib

Repository metrics

Stars
 (21,837 stars)
PR merge metrics
 (Avg merge 5d 12h) (85 merged PRs in 30d)

Description

We have a Kafka topic with msgpack events we would like to deserialize (and reserialize after some transformations in an other topic). Currently, we use Logstash for that with this plugin : https://github.com/logstash-plugins/logstash-codec-msgpack

It could be very useful to have a generic msgpack codec in Vector like for JSON format, maybe in the form of a pair VRL functions parse_msgpack / encode_msgpack ?

Contributor guide

Research direction
Study existing codec implementations in Vector (e.g., JSON codec) and VRL function patterns (parse json/encode json) to implement MessagePack codec or VRL functions.
Tech stack
rust
Domain
backendapidata
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
RustVector architecture
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
30