Repository navigation
Conversation
…k with `from_value` Serialize `data` with padded STANDARD base64. Deserialize decodes with STANDARD, which requires canonical padding since base64 0.20, so the unpadded output failed to deserialize for about two payloads in three. Read the map key through a field-identifier enum instead of `&str`, so deserializers that cannot lend borrowed strings (`serde_json::from_value`, `from_reader`) work too.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
📬 Issue #, if available:
N/A
✍️ Description of changes:
cloudwatch_logs::AwsLogs(theawslogsfield ofLogsEvent) has hand-writtenSerialize/Deserializeimpls, and they have two problems.1. Serialized
datais unpadded base64 that the crate's own deserializer rejects.Serializeencodes withSTANDARD_NO_PAD, butDeserializedecodes withSTANDARD. Since base64 0.20,STANDARDrequires canonical padding (base64 0.13's decoder ignored padding; the crate moved from 0.13 to 0.21 in #650). Soserde_json::to_string(&event)followed byserde_json::from_str::<LogsEvent>fails withInvalid paddingwhenever the gzip output is not a multiple of 3 bytes long, which is about two payloads in three. The two existing round-trip tests only pass because both of their payloads happen to compress to a multiple of 3 bytes (re-serializeddatalengths 276 and 496). Other consumers of this format reject the unpadded output too: Go'sbase64.StdEncoding(which aws-lambda-go'sCloudwatchLogsRawData.Parseuses) fails withillegal base64 data at input byte 280, and Python'sbase64.b64decodefails withIncorrect padding. The incoming payloads are padded. This crate's ownexample-cloudwatch_logs-event.jsonfixture ends in==, and so does aws-lambda-go'scloudwatch-logs-event.jsontestdata. The captured event intest_deserialize_exampleends in=. RFC 4648 §3.2 also requires padding unless the referring spec says otherwise.2.
Deserializeonly works with deserializers that can lend borrowed strings. The visitor reads the map key as&str(map.next_key()?matched against"data").serde_json::from_slice/from_str, whichlambda_runtimeuses, can lend borrowed strings, so direct invocations work.serde_json::from_valueandserde_json::from_readercannot, and both fail withinvalid type: string "data", expected a borrowed string. The other hand-written visitors in the crate (custom_serde/headers.rsandiam) read their keys asCow<str>, which works with both APIs;AwsLogsis the only one that borrows its map key. This breaks:LambdaEvent<serde_json::Value>and pick the concrete type withserde_json::from_value, a common pattern when one function is subscribed to several sources;serde_json::json!+from_value;from_reader.Changes:
datawithSTANDARD, so the output is padded;#[serde(field_identifier)]enum instead of&str. It accepts borrowed and owned keys without allocating, and the error messages are unchanged (unknown field `datax`, expected `data`andmissing field `data`). In a release-mode timing of 100kfrom_slicecalls on the fixture (11 interleaved runs, best of 5 each), the median was 4779 ns per event on main and 4722 ns with this change. Reading the key as an ownedStringinstead measured 4823 ns, which is why I used the enum.Tests:
example_cloudwatch_logs_event_from_value_and_reader: deserializes the existing fixture throughfrom_valueandfrom_readerand compares the results withfrom_slice.test_serialize_round_trip_pads_base64: serializes events of growing message length, checks that each one deserializes back to an equal value and thatdatadecodes withSTANDARD. It also asserts that at least one payload length was not a multiple of 3, so the test cannot pass by luck.Both tests fail on main:
With this change both tests pass:
I ran these commands on macOS arm64 (stable Rust, nightly rustfmt):
cargo test -p aws_lambda_events --all-featurescargo test -p aws_lambda_eventscargo test -p aws_lambda_events --no-default-features --features cloudwatch_logscargo +nightly fmt --all -- --checkcargo clippy -p aws_lambda_events --all-features -- -D warningsAll of them pass.
🔏 By submitting this pull request
cargo +nightly fmt.cargo clippy --fix.This change was drafted with Claude Code (claude-opus-5-5); the test results above are from the runs described.