Use tar.Header references over copies in tar index

[wagoodman]

This modifies the image metadata and tar index to always use tar.Header references over copies to slightly lower inuse memory utilization. Additionally this removes the sequence field altogether as it is not used.

Inuse memory before the change:

Inuse memory after the change:

Partially addresses #233

[github-actions]

Benchmark Test Results

Benchmark results from the latest changes vs base branch

make .tool/task
make[1]: Entering directory '/home/runner/work/stereoscope/stereoscope'
make[1]: Leaving directory '/home/runner/work/stereoscope/stereoscope'
.tool/task show-benchstat
?   	github.com/anchore/stereoscope	[no test files]
?   	github.com/anchore/stereoscope/examples	[no test files]
PASS
ok  	github.com/anchore/stereoscope/internal	0.004s
?   	github.com/anchore/stereoscope/internal/bus	[no test files]
PASS
ok  	github.com/anchore/stereoscope/internal/containerd	0.008s
PASS
ok  	github.com/anchore/stereoscope/internal/docker	0.005s
?   	github.com/anchore/stereoscope/internal/log	[no test files]
PASS
ok  	github.com/anchore/stereoscope/internal/podman	0.005s
?   	github.com/anchore/stereoscope/pkg/event	[no test files]
?   	github.com/anchore/stereoscope/pkg/event/parsers	[no test files]
goos: linux
goarch: amd64
pkg: github.com/anchore/stereoscope/pkg/file
cpu: AMD EPYC 7763 64-Core Processor                
BenchmarkTarIndex-4   	   33872	     35367 ns/op	    4128 B/op	      93 allocs/op
BenchmarkTarIndex-4   	   33831	     35500 ns/op	    4130 B/op	      93 allocs/op
BenchmarkTarIndex-4   	   33634	     35570 ns/op	    4129 B/op	      93 allocs/op
BenchmarkTarIndex-4   	   33900	     35471 ns/op	    4129 B/op	      93 allocs/op
BenchmarkTarIndex-4   	   34008	     36474 ns/op	    4129 B/op	      93 allocs/op
BenchmarkTarIndex-4   	   33427	     35241 ns/op	    4128 B/op	      93 allocs/op
BenchmarkTarIndex-4   	   33894	     35183 ns/op	    4127 B/op	      93 allocs/op
PASS
ok  	github.com/anchore/stereoscope/pkg/file	10.939s
PASS
ok  	github.com/anchore/stereoscope/pkg/filetree	0.005s
?   	github.com/anchore/stereoscope/pkg/filetree/filenode	[no test files]
PASS
ok  	github.com/anchore/stereoscope/pkg/image	0.006s
PASS
ok  	github.com/anchore/stereoscope/pkg/image/containerd	0.009s
PASS
ok  	github.com/anchore/stereoscope/pkg/image/docker	0.006s
PASS
ok  	github.com/anchore/stereoscope/pkg/image/oci	0.006s
PASS
ok  	github.com/anchore/stereoscope/pkg/image/oci/credhelpers	0.005s
?   	github.com/anchore/stereoscope/pkg/image/podman	[no test files]
PASS
ok  	github.com/anchore/stereoscope/pkg/image/sif	0.005s
?   	github.com/anchore/stereoscope/pkg/imagetest	[no test files]
PASS
ok  	github.com/anchore/stereoscope/pkg/tree	0.004s
PASS
ok  	github.com/anchore/stereoscope/pkg/tree/node	0.004s
goos: linux
goarch: amd64
pkg: github.com/anchore/stereoscope/test/integration
cpu: AMD EPYC 7763 64-Core Processor                
BenchmarkSimpleImage_GetImage/docker-archive-4 	    1096	   1066620 ns/op	  276076 B/op	    2186 allocs/op
BenchmarkSimpleImage_GetImage/docker-archive-4 	    1123	   1062528 ns/op	  275669 B/op	    2185 allocs/op
BenchmarkSimpleImage_GetImage/docker-archive-4 	    1129	   1063538 ns/op	  275562 B/op	    2185 allocs/op
BenchmarkSimpleImage_GetImage/docker-archive-4 	    1118	   1055955 ns/op	  275466 B/op	    2185 allocs/op
BenchmarkSimpleImage_GetImage/docker-archive-4 	     939	   1157617 ns/op	  275492 B/op	    2185 allocs/op
BenchmarkSimpleImage_GetImage/docker-archive-4 	    1110	   1123801 ns/op	  275309 B/op	    2185 allocs/op
BenchmarkSimpleImage_GetImage/docker-archive-4 	    1116	   1150876 ns/op	  275245 B/op	    2185 allocs/op
BenchmarkSimpleImage_GetImage/podman-4         	      62	  17668575 ns/op	  406247 B/op	    2677 allocs/op
BenchmarkSimpleImage_GetImage/podman-4         	      64	  17715590 ns/op	  405850 B/op	    2676 allocs/op
BenchmarkSimpleImage_GetImage/podman-4         	      72	  17470192 ns/op	  405914 B/op	    2675 allocs/op
BenchmarkSimpleImage_GetImage/podman-4         	      69	  20634683 ns/op	  406019 B/op	    2675 allocs/op
BenchmarkSimpleImage_GetImage/podman-4         	      64	  17609625 ns/op	  406214 B/op	    2675 allocs/op
BenchmarkSimpleImage_GetImage/podman-4         	      66	  17797945 ns/op	  405505 B/op	    2674 allocs/op
BenchmarkSimpleImage_GetImage/podman-4         	      63	  17461178 ns/op	  405163 B/op	    2674 allocs/op
#0 building with "default" instance using docker driver

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 345B done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 transferring context: 2B done
#2 DONE 0.0s

#3 [internal] load build context
#3 transferring context: 209B done
#3 DONE 0.0s

#4 [2/3] ADD file-2.txt /somefile-2.txt
#4 CACHED

#5 [1/3] ADD file-1.txt /somefile-1.txt
#5 CACHED

#6 [3/3] ADD target /
#6 CACHED

#7 exporting to image
#7 exporting layers done
#7 writing image sha256:85eaa6e740d88ff4fb7926f34bd6f9c330a97218431acfa27fff8087146478fa done
#7 naming to docker.io/library/stereoscope-fixture-image-simple:04e16e44161c8888a1a963720fd0443cbf7eef8101434c431de8725cd98cc9f7 done
#7 naming to docker.io/library/stereoscope-fixture-image-simple:latest done
#7 DONE 0.0s
ctr: failed to dial "/run/containerd/containerd.sock": connection error: desc = "transport: error while dialing: dial unix /run/containerd/containerd.sock: connect: permission denied"
--- FAIL: BenchmarkSimpleImage_GetImage
    image_fixtures.go:193: using existing image tar: 'test-fixtures/cache/stereoscope-fixture-image-simple-04e16e44161c8888a1a963720fd0443cbf7eef8101434c431de8725cd98cc9f7.tar' (size: 22528, modified: 2024-07-11 13:21:55.046908121 +0000 UTC, mode: -rw-r--r--)
    image_fixtures.go:241: Build docker image: name="stereoscope-fixture-image-simple" tag="04e16e44161c8888a1a963720fd0443cbf7eef8101434c431de8725cd98cc9f7"
    image_fixtures.go:291: saveImage running: docker image save stereoscope-fixture-image-simple:04e16e44161c8888a1a963720fd0443cbf7eef8101434c431de8725cd98cc9f7
    image_fixtures.go:286: 
        	Error Trace:	/home/runner/work/stereoscope/stereoscope/pkg/imagetest/image_fixtures.go:286
        	            				/home/runner/work/stereoscope/stereoscope/pkg/imagetest/image_fixtures.go:162
        	            				/home/runner/work/stereoscope/stereoscope/pkg/imagetest/image_fixtures.go:152
        	            				/home/runner/work/stereoscope/stereoscope/pkg/imagetest/image_fixtures.go:33
        	            				/home/runner/work/stereoscope/stereoscope/test/integration/fixture_image_simple_test.go:163
        	Error:      	Received unexpected error:
        	            	exit status 1
        	Test:       	BenchmarkSimpleImage_GetImage
        	Messages:   	could not import docker image to containerd (shell out)
BenchmarkSimpleImage_FetchSquashedContents/docker-archive-4         	   53944	     22071 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/docker-archive-4         	   53629	     22120 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/docker-archive-4         	   53712	     22127 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/docker-archive-4         	   54345	     22094 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/docker-archive-4         	   53748	     22070 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/docker-archive-4         	   53962	     22046 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/docker-archive-4         	   54121	     22107 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/podman-4                 	   53833	     22175 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/podman-4                 	   53628	     22187 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/podman-4                 	   54064	     22042 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/podman-4                 	   54116	     22186 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/podman-4                 	   54070	     22153 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/podman-4                 	   53467	     22259 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/podman-4                 	   53568	     22240 ns/op	    2712 B/op	      21 allocs/op
#0 building with "default" instance using docker driver

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 345B done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 transferring context: 2B done
#2 DONE 0.0s

#3 [internal] load build context
#3 transferring context: 209B done
#3 DONE 0.0s

#4 [1/3] ADD file-1.txt /somefile-1.txt
#4 CACHED

#5 [2/3] ADD file-2.txt /somefile-2.txt
#5 CACHED

#6 [3/3] ADD target /
#6 CACHED

#7 exporting to image
#7 exporting layers done
#7 writing image sha256:85eaa6e740d88ff4fb7926f34bd6f9c330a97218431acfa27fff8087146478fa done
#7 naming to docker.io/library/stereoscope-fixture-image-simple:04e16e44161c8888a1a963720fd0443cbf7eef8101434c431de8725cd98cc9f7 done
#7 naming to docker.io/library/stereoscope-fixture-image-simple:latest done
#7 DONE 0.0s
ctr: failed to dial "/run/containerd/containerd.sock": connection error: desc = "transport: error while dialing: dial unix /run/containerd/containerd.sock: connect: permission denied"
--- FAIL: BenchmarkSimpleImage_FetchSquashedContents
    image_fixtures.go:193: using existing image tar: 'test-fixtures/cache/stereoscope-fixture-image-simple-04e16e44161c8888a1a963720fd0443cbf7eef8101434c431de8725cd98cc9f7.tar' (size: 22528, modified: 2024-07-11 13:21:55.046908121 +0000 UTC, mode: -rw-r--r--)
    image_fixtures.go:241: Build docker image: name="stereoscope-fixture-image-simple" tag="04e16e44161c8888a1a963720fd0443cbf7eef8101434c431de8725cd98cc9f7"
    image_fixtures.go:291: saveImage running: docker image save stereoscope-fixture-image-simple:04e16e44161c8888a1a963720fd0443cbf7eef8101434c431de8725cd98cc9f7
    image_fixtures.go:286: 
        	Error Trace:	/home/runner/work/stereoscope/stereoscope/pkg/imagetest/image_fixtures.go:286
        	            				/home/runner/work/stereoscope/stereoscope/pkg/imagetest/image_fixtures.go:162
        	            				/home/runner/work/stereoscope/stereoscope/pkg/imagetest/image_fixtures.go:152
        	            				/home/runner/work/stereoscope/stereoscope/pkg/imagetest/image_fixtures.go:33
        	            				/home/runner/work/stereoscope/stereoscope/pkg/imagetest/image_fixtures.go:64
        	            				/home/runner/work/stereoscope/stereoscope/test/integration/fixture_image_simple_test.go:189
        	Error:      	Received unexpected error:
        	            	exit status 1
        	Test:       	BenchmarkSimpleImage_FetchSquashedContents
        	Messages:   	could not import docker image to containerd (shell out)
FAIL
exit status 1
FAIL	github.com/anchore/stereoscope/test/integration	38.719s
?   	github.com/anchore/stereoscope/test/integration/test-fixtures/registry	[no test files]
FAIL
goos: linux
goarch: amd64
pkg: github.com/anchore/stereoscope/pkg/file
cpu: AMD EPYC 7763 64-Core Processor                
ctr: 
           │ .tmp/benchmark-db573f4.txt │
           │           sec/op           │
TarIndex-4                  35.47µ ± 3%

           │ .tmp/benchmark-db573f4.txt │
           │            B/op            │
TarIndex-4                 4.032Ki ± 0%

           │ .tmp/benchmark-db573f4.txt │
           │         allocs/op          │
TarIndex-4                   93.00 ± 0%

pkg: github.com/anchore/stereoscope/test/integration
                                      │ .tmp/benchmark-db573f4.txt │
                                      │           sec/op           │
SimpleImage_GetImage/docker-archive-4                 1.067m ±  9%
SimpleImage_GetImage/podman-4                         17.67m ± 17%
geomean                                               4.341m

                                      │ .tmp/benchmark-db573f4.txt │
                                      │            B/op            │
SimpleImage_GetImage/docker-archive-4                 269.0Ki ± 0%
SimpleImage_GetImage/podman-4                         396.4Ki ± 0%
geomean                                               326.6Ki

                                      │ .tmp/benchmark-db573f4.txt │
                                      │         allocs/op          │
SimpleImage_GetImage/docker-archive-4                  2.185k ± 0%
SimpleImage_GetImage/podman-4                          2.675k ± 0%
geomean                                                2.418k

ctr: failed to dial "/run/containerd/containerd.sock": connection error: desc = "transport: error while dialing: dial unix /run/containerd/containerd.sock: connect: permission denied"
                                                   │ .tmp/benchmark-db573f4.txt │
                                                   │           sec/op           │
SimpleImage_FetchSquashedContents/docker-archive-4                  22.09µ ± 0%
SimpleImage_FetchSquashedContents/podman-4                          22.19µ ± 1%
geomean                                                             22.14µ

                                                   │ .tmp/benchmark-db573f4.txt │
                                                   │            B/op            │
SimpleImage_FetchSquashedContents/docker-archive-4                 2.648Ki ± 0%
SimpleImage_FetchSquashedContents/podman-4                         2.648Ki ± 0%
geomean                                                            2.648Ki

                                                   │ .tmp/benchmark-db573f4.txt │
                                                   │         allocs/op          │
SimpleImage_FetchSquashedContents/docker-archive-4                   21.00 ± 0%
SimpleImage_FetchSquashedContents/podman-4                           21.00 ± 0%
geomean                                                              21.00
goos: linux
goarch: amd64
pkg: github.com/anchore/stereoscope/pkg/file
cpu: AMD EPYC 7763 64-Core Processor                
ctr: 
           │ .tmp/benchmark-db573f4.txt │
           │           sec/op           │
TarIndex-4                  35.47µ ± 3%

           │ .tmp/benchmark-db573f4.txt │
           │            B/op            │
TarIndex-4                 4.032Ki ± 0%

           │ .tmp/benchmark-db573f4.txt │
           │         allocs/op          │
TarIndex-4                   93.00 ± 0%

pkg: github.com/anchore/stereoscope/test/integration
                                      │ .tmp/benchmark-db573f4.txt │
                                      │           sec/op           │
SimpleImage_GetImage/docker-archive-4                 1.067m ±  9%
SimpleImage_GetImage/podman-4                         17.67m ± 17%
geomean                                               4.341m

                                      │ .tmp/benchmark-db573f4.txt │
                                      │            B/op            │
SimpleImage_GetImage/docker-archive-4                 269.0Ki ± 0%
SimpleImage_GetImage/podman-4                         396.4Ki ± 0%
geomean                                               326.6Ki

                                      │ .tmp/benchmark-db573f4.txt │
                                      │         allocs/op          │
SimpleImage_GetImage/docker-archive-4                  2.185k ± 0%
SimpleImage_GetImage/podman-4                          2.675k ± 0%
geomean                                                2.418k

ctr: failed to dial "/run/containerd/containerd.sock": connection error: desc = "transport: error while dialing: dial unix /run/containerd/containerd.sock: connect: permission denied"
                                                   │ .tmp/benchmark-db573f4.txt │
                                                   │           sec/op           │
SimpleImage_FetchSquashedContents/docker-archive-4                  22.09µ ± 0%
SimpleImage_FetchSquashedContents/podman-4                          22.19µ ± 1%
geomean                                                             22.14µ

                                                   │ .tmp/benchmark-db573f4.txt │
                                                   │            B/op            │
SimpleImage_FetchSquashedContents/docker-archive-4                 2.648Ki ± 0%
SimpleImage_FetchSquashedContents/podman-4                         2.648Ki ± 0%
geomean                                                            2.648Ki

                                                   │ .tmp/benchmark-db573f4.txt │
                                                   │         allocs/op          │
SimpleImage_FetchSquashedContents/docker-archive-4                   21.00 ± 0%
SimpleImage_FetchSquashedContents/podman-4                           21.00 ± 0%
geomean                                                              21.00