Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

optimize the manifest conversion from bytes to unstructured #7577

Open
Tracked by #7532
smira opened this issue Aug 4, 2023 · 1 comment
Open
Tracked by #7532

optimize the manifest conversion from bytes to unstructured #7577

smira opened this issue Aug 4, 2023 · 1 comment

Comments

@smira
Copy link
Member

smira commented Aug 4, 2023

See the code in

func (a manifest) SetYAML(yamlBytes []byte) error {
a.Manifest.TypedSpec().Items = nil
reader := yaml.NewYAMLReader(bufio.NewReader(bytes.NewReader(yamlBytes)))
for {
yamlManifest, err := reader.Read()
if err != nil {
if err == io.EOF {
break
}
return err
}
yamlManifest = bytes.TrimSpace(yamlManifest)
if len(yamlManifest) == 0 {
continue
}
jsonManifest, err := yaml.ToJSON(yamlManifest)
if err != nil {
return fmt.Errorf("error converting manifest to JSON: %w", err)
}
if bytes.Equal(jsonManifest, []byte("null")) || bytes.Equal(jsonManifest, []byte("{}")) {
// skip YAML docs which contain only comments
continue
}
var obj unstructured.Unstructured
if err = json.Unmarshal(jsonManifest, &obj); err != nil {
return fmt.Errorf("error loading JSON manifest into unstructured: %w", err)
}
a.Manifest.TypedSpec().Items = append(a.Manifest.TypedSpec().Items, k8s.SingleManifest{Object: obj.Object})
}
return nil
}

There are many copies over:

  • first the YAML document is split into sub-documents with inefficient bufio
  • second, the YAML is re-marshalled to JSON
  • third, JSON is unmarshalled to object.Object

We could look if we could go directly from YAML to unstructured.Unstructured (performing all required conversions on the fly).

Copy link

This issue is stale because it has been open 180 days with no activity. Remove stale label or comment or this will be closed in 7 days.

@github-actions github-actions bot added the Stale label Jun 20, 2024
@DmitriyMV DmitriyMV removed the Stale label Jun 20, 2024
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants