repo reorganisation

[Dieterbe]

our repo layout is a bit of a mess. my main issues:

1. hard to differentiate go packages from other dirs. e.g. conf is a go package whereas scripts/config is where config files live. it's confusing because almost all directories are go packages, but not all (e.g. docker, docs, devdocs, dashboards, etc)
2. in the go community, any exported code is assumed to be reusable by anyone, anywhere. we don't have any stability guarantees at the exported golang api level. We have been exporting any code as soon as we need to access it from any other package (internal in the metrictank repo, or any of our other projects, e.g. github.com/raintank/tsdb-gw or github.com/graphite-ng/carbon-relay-ng), we probably even have exported code that doesn't need to be exported at all.
3. in particular, we don't follow some common best practices:

• we don't have an internal directory, which enables us to forbid any non-MT importing any of our code if it lives under this directory, even if it's exported. (personally i find it a bit too agressive, i rather just document we don't provide guarantees at the api level. also, implementing this, would probably cause breakage. e.g. in carbon-relay-ng or tsdbgw)
• we don't have a pkg directory. this one seems a bit contentious:

• pkg directory should not be recommended for use golang-standards/project-layout#10 : against: bradfitz, in favor: peter bourgon. abstain: JBD (note, older version of peter's thoughts on this - before "internal" was a thing (https://peter.bourgon.org/go-best-practices-2016/#top-tip-2))
• https://medium.com/golang-learn/go-project-layout-e5213cdcfaa2 is in favor, though recommends "internal" for internal packages
• and then there is https://medium.com/@benbjohnson/standard-package-layout-7cdbc8391fc1, which is a very interesting idea, one perhaps we're not ready for either, and it doesn't mention pkg or internal anyway.

I note that grafana/grafana, cortextproject/cortex and prometheus/prometheus have "pkg" dirs
prometheus has both a pkg directory but also a bunch of packages in the root directory (includirg exported stuff like in rules/alerting.go). not sure why.

My suggestion is:

• we adopt "pkg" for all of our non-main packages (internal or otherwise). some people consider it deprecated but i think it would clarify our repo layout quite a bit.
• we don't implement "internal" to keep our life simple
• any further reorganization can be done in addition on a case by case basis, or as an additional step later, once we get this change through.

thoughts?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.