Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

[storage] [lint] Start enforcing lint errors #197

Merged
merged 2 commits into from
Nov 9, 2022
Merged

[storage] [lint] Start enforcing lint errors #197

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c6c60bd
[storage] [lint] Start enforcing lint errors
glimchb Nov 8, 2022
00067a0
[storage] [lint] Enforcing lint errors on backend
glimchb Nov 9, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions storage/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,6 @@ all:
curl -kL https://github.com/googleapis/googleapis/archive/master.tar.gz | tar --strip=1 -zxvf - googleapis-master/google/api
mv google "${PWD}"/v1alpha1/
docker run --user=$$(id -u):$$(id -g) --rm --entrypoint=sh -v "${PWD}/../common/v1":/common -v "${PWD}"/v1alpha1/:/out -w /out ghcr.io/docker-multiarch/google-api-linter:1.36.0 -c "api-linter -I /common /out/*.proto --output-format summary"
docker run --user=$$(id -u):$$(id -g) --rm --entrypoint=sh -v "${PWD}/../common/v1":/common -v "${PWD}"/v1alpha1/:/out -w /out ghcr.io/docker-multiarch/google-api-linter:1.36.0 -c "api-linter -I /common /out/*.proto --output-format github --disable-rule=core::0192::has-comments --disable-rule=core::0136 --disable-rule=core::0127 --set-exit-status"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be nice to document the rules being ignored and why. But not a blocker.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just wanted to start enforcing rules, so ignore is temp thing, eventually nothing will be ignored, thanks @seroyer

docker run --user=$$(id -u):$$(id -g) --rm --entrypoint=sh -v "${PWD}/../common/v1":/common -v "${PWD}"/v1alpha1/:/out -w /out -v "${PWD}"/v1alpha1:/protos pseudomuto/protoc-gen-doc -c "protoc -I /common -I /protos --doc_out=/out --doc_opt=markdown,autogen.md /protos/*.proto /common/*.proto"
rm -rf "${PWD}"/v1alpha1/google
61 changes: 30 additions & 31 deletions storage/v1alpha1/autogen.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -154,9 +154,8 @@
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
| handle | [opi_api.common.v1.ObjectKey](#opi_api-common-v1-ObjectKey) | | handle is an opaque object handle that is not user settable. handle will be returned with created object |
| name | [string](#string) | | |
| block_size | [int64](#int64) | | |
| num_blocks | [int64](#int64) | | |
| blocks_count | [int64](#int64) | | |
| uuid | [opi_api.common.v1.Uuid](#opi_api-common-v1-Uuid) | | |
| filename | [string](#string) | | |

Expand Down Expand Up @@ -305,7 +304,7 @@ Intentionally empty
<a name="opi_api-storage-v1-AioControllerService"></a>

### AioControllerService

Back End (network-facing) APIs. This service is for AIO generic kernel block device.

| Method Name | Request Type | Response Type | Description |
| ----------- | ------------ | ------------- | ------------|
Expand Down Expand Up @@ -352,9 +351,8 @@ Intentionally empty
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
| id | [int64](#int64) | | |
| name | [string](#string) | | |
| block_size | [int64](#int64) | | |
| num_blocks | [int64](#int64) | | |
| blocks_count | [int64](#int64) | | |
| uuid | [opi_api.common.v1.Uuid](#opi_api-common-v1-Uuid) | | |


Expand Down Expand Up @@ -532,7 +530,7 @@ Intentionally empty.
<a name="opi_api-storage-v1-NullDebugService"></a>

### NullDebugService

Back End (network-facing) APIs. This is debug interface for null block devices.

| Method Name | Request Type | Response Type | Description |
| ----------- | ------------ | ------------- | ------------|
Expand Down Expand Up @@ -587,7 +585,7 @@ Intentionally empty.
| hdgst | [bool](#bool) | | |
| ddgst | [bool](#bool) | | |
| multipath | [NvmeMultipath](#opi_api-storage-v1-NvmeMultipath) | | |
| num_io_queues | [int64](#int64) | | |
| io_queues_count | [int64](#int64) | | |
| queue_size | [int64](#int64) | | |


Expand Down Expand Up @@ -770,11 +768,12 @@ Intentionally empty.

| Name | Number | Description |
| ---- | ------ | ----------- |
| NVMF_ADRFAM_IPV4 | 0 | |
| NVMF_ADRFAM_IPV6 | 1 | |
| NVMF_ADRFAM_IB | 2 | |
| NVMF_ADRFAM_FC | 3 | |
| NVMF_ADRFAM_INTRA_HOST | 4 | |
| NVME_ADDRESS_FAMILY_UNSPECIFIED | 0 | |
| NVMF_ADRFAM_IPV4 | 1 | |
| NVMF_ADRFAM_IPV6 | 2 | |
| NVMF_ADRFAM_IB | 3 | |
| NVMF_ADRFAM_FC | 4 | |
| NVMF_ADRFAM_INTRA_HOST | 5 | |



Expand All @@ -785,9 +784,10 @@ Intentionally empty.

| Name | Number | Description |
| ---- | ------ | ----------- |
| NVME_MPIO_DISABLE | 0 | |
| NVME_MPIO_FAILOVER | 1 | |
| NVME_MPIO_MULTIPATH | 2 | |
| NVME_MULTIPATH_UNSPECIFIED | 0 | |
| NVME_MULTIPATH_DISABLE | 1 | |
| NVME_MULTIPATH_FAILOVER | 2 | |
| NVME_MULTIPATH_MULTIPATH | 3 | |



Expand All @@ -798,11 +798,12 @@ Intentionally empty.

| Name | Number | Description |
| ---- | ------ | ----------- |
| NVME_TRANSPORT_FC | 0 | |
| NVME_TRANSPORT_PCIE | 1 | |
| NVME_TRANSPORT_RDMA | 2 | |
| NVME_TRANSPORT_TCP | 3 | |
| NVME_TRANSPORT_CUSTOM | 4 | |
| NVME_TRANSPORT_TYPE_UNSPECIFIED | 0 | |
| NVME_TRANSPORT_FC | 1 | |
| NVME_TRANSPORT_PCIE | 2 | |
| NVME_TRANSPORT_RDMA | 3 | |
| NVME_TRANSPORT_TCP | 4 | |
| NVME_TRANSPORT_CUSTOM | 5 | |



Expand All @@ -813,7 +814,7 @@ Intentionally empty.
<a name="opi_api-storage-v1-NVMfRemoteControllerService"></a>

### NVMfRemoteControllerService

Back End (network-facing) APIs. NVMe/TCP and NVMe/RoCEv2 protocols are covered by this service.

| Method Name | Request Type | Response Type | Description |
| ----------- | ------------ | ------------- | ------------|
Expand Down Expand Up @@ -849,7 +850,7 @@ virtual functions under the physical function.
| ----- | ---- | ----- | ----------- |
| bus | [int32](#int32) | | Bus number, provided for future usage if needed. Currently set to ’0’ |
| device | [int32](#int32) | | Device number, based on the NVMe device layout |
| function | [int32](#int32) | | Physical function, always set to 0 in current model |
| func | [int32](#int32) | | Physical function, always set to 0 in current model |
| virtual_function | [int32](#int32) | | SRIOV Virtual function within the Device and Physical function. Set to 0 for Physical Function. Virtual Function numbering starts from 1 |


Expand Down Expand Up @@ -886,7 +887,7 @@ virtual functions under the physical function.
| subsystem_id | [opi_api.common.v1.ObjectKey](#opi_api-common-v1-ObjectKey) | | subsystem information |
| pcie_id | [NvmeControllerPciId](#opi_api-storage-v1-NvmeControllerPciId) | | xPU&#39;s PCI ID for the controller |
| max_nsq | [int32](#int32) | | maximum number of host submission queues allowed. If not set, the xPU will provide a default. |
| max_ncq | [int32](#int32) | | maximum number of host completion queues allowed. Optional. If not set, the xPU will provide a default. |
| max_ncq | [int32](#int32) | | maximum number of host completion queues allowed. If not set, the xPU will provide a default. |
| sqes | [int32](#int32) | | maximum number of submission queue entries per submission queue, as a power of 2. default value as per spec is 6 |
| cqes | [int32](#int32) | | maximum number of completion queue entries per completion queue, as a power of 2. default value as per spec is 4 |
| max_namespaces | [int32](#int32) | | maximum Number of namespaces that will be provisioned under the controller. |
Expand Down Expand Up @@ -1025,14 +1026,14 @@ virtual functions under the physical function.

| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
| id | [opi_api.common.v1.ObjectKey](#opi_api-common-v1-ObjectKey) | | namespace&#39;s unique key replaces: int64 id = 1; |
| id | [opi_api.common.v1.ObjectKey](#opi_api-common-v1-ObjectKey) | | namespace&#39;s unique key |
| subsystem_id | [opi_api.common.v1.ObjectKey](#opi_api-common-v1-ObjectKey) | | subsystem for this namespace |
| controller_id | [opi_api.common.v1.ObjectKey](#opi_api-common-v1-ObjectKey) | | key of the PCIe controller object that will host this namespace. |
| host_nsid | [int32](#int32) | | NSID present to the host by the NVMe PCIe controller. If not provided, then the controller will assign an unused NSID within the max namespace range - auto assigned nsid may not work for live migration |
| block_size | [int64](#int64) | | Block size in bytes, must be power of 2 and must be less than the max io size supported. Typically tested values are 512, and 4k. |
| num_blocks | [int64](#int64) | | Size/Capacity of the namespace in blocks, size in bytes will be BlockSize x NumBlocks. |
| blocks_count | [int64](#int64) | | Size/Capacity of the namespace in blocks, size in bytes will be BlockSize x NumBlocks. |
| nguid | [string](#string) | | Globally unique identifier for the namespace |
| eui64 | [int64](#int64) | | 64bit Extended unique identifier for the namespace mandatory if guid is not specified, optional otherwise |
| eui64 | [int64](#int64) | | 64bit Extended unique identifier for the namespace mandatory if guid is not specified |
| uuid | [opi_api.common.v1.Uuid](#opi_api-common-v1-Uuid) | | Globally unique identifier for the namespace |
| volume_id | [opi_api.common.v1.ObjectKey](#opi_api-common-v1-ObjectKey) | | The back/middle-end volume to back this namespace. |
| optimal_write_size | [int32](#int32) | | optimal write size hint to host driver. Host IO stack may use this to regulate IO size. Must be a multiple of the preferred write granularity. Must not exceed the controller maximum IO size value configured in the nvme agent config file. |
Expand Down Expand Up @@ -1310,7 +1311,7 @@ Intentionally empty.
<a name="opi_api-storage-v1-FrontendNvmeService"></a>

### FrontendNvmeService

Front End (host-facing) APIs. Mostly used for NVMe/PCIe emulation and host presentation.

| Method Name | Request Type | Response Type | Description |
| ----------- | ------------ | ------------- | ------------|
Expand Down Expand Up @@ -1353,7 +1354,6 @@ Intentionally empty.
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
| id | [int64](#int64) | | |
| name | [string](#string) | | |
| pcie_id | [NvmeControllerPciId](#opi_api-storage-v1-NvmeControllerPciId) | | |
| bdev | [string](#string) | | |
| max_io_qps | [int64](#int64) | | |
Expand Down Expand Up @@ -1494,7 +1494,7 @@ Intentionally empty.
<a name="opi_api-storage-v1-FrontendVirtioBlkService"></a>

### FrontendVirtioBlkService

Front End (host-facing) APIs. Mostly used for Virtio-blk emulation emulation and host presentation as alternative to Nvme.

| Method Name | Request Type | Response Type | Description |
| ----------- | ------------ | ------------- | ------------|
Expand Down Expand Up @@ -1541,7 +1541,6 @@ Intentionally empty.
| Field | Type | Label | Description |
| ----- | ---- | ----- | ----------- |
| id | [int64](#int64) | | |
| name | [string](#string) | | |
| pcie_id | [NvmeControllerPciId](#opi_api-storage-v1-NvmeControllerPciId) | | |


Expand Down Expand Up @@ -1815,7 +1814,7 @@ Intentionally empty.
<a name="opi_api-storage-v1-FrontendVirtioScsiService"></a>

### FrontendVirtioScsiService

Front End (host-facing) APIs. Mostly used for Virtio-scsi emulation emulation and host presentation as alternative to Virtio-blk.

| Method Name | Request Type | Response Type | Description |
| ----------- | ------------ | ------------- | ------------|
Expand Down
12 changes: 5 additions & 7 deletions storage/v1alpha1/backend_aio.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,7 @@ import "google/protobuf/empty.proto";
import "object_key.proto";
import "uuid.proto";

// Back End (network-facing) APIs.

// Back End (network-facing) APIs. This service is for AIO generic kernel block device.
service AioControllerService {
rpc AioControllerCreate (AioControllerCreateRequest) returns (AioController) {}
rpc AioControllerDelete (AioControllerDeleteRequest) returns (google.protobuf.Empty) {}
Expand All @@ -30,11 +29,10 @@ message AioController {
// handle will be returned with created object
common.v1.ObjectKey handle = 1;

string name = 2;
int64 block_size = 3;
int64 num_blocks = 4;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://linter.aip.dev/141/count-suffix

common.v1.Uuid uuid =5;
string filename = 6;
int64 block_size = 2;
int64 blocks_count = 3;
common.v1.Uuid uuid = 4;
string filename = 5;
}

message AioControllerCreateRequest {
Expand Down
10 changes: 4 additions & 6 deletions storage/v1alpha1/backend_null.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,7 @@ option go_package = "github.com/opiproject/opi-api/storage/v1alpha1/gen/go";
import "object_key.proto";
import "uuid.proto";

// Back End (network-facing) APIs.

// Back End (network-facing) APIs. This is debug interface for null block devices.
service NullDebugService {
rpc NullDebugCreate (NullDebugCreateRequest) returns (NullDebugCreateResponse) {}
rpc NullDebugDelete (NullDebugDeleteRequest) returns (NullDebugDeleteResponse) {}
Expand All @@ -25,10 +24,9 @@ service NullDebugService {

message NullDebug {
int64 id = 1;
string name = 2;
int64 block_size = 3;
int64 num_blocks = 4;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://linter.aip.dev/141/count-suffix

common.v1.Uuid uuid = 5;
int64 block_size = 2;
int64 blocks_count = 3;
common.v1.Uuid uuid = 4;
}

message NullDebugCreateRequest {
Expand Down
54 changes: 26 additions & 28 deletions storage/v1alpha1/backend_nvme_tcp.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,7 @@ option java_outer_classname = "BackendNvmeTcpProto";

option go_package = "github.com/opiproject/opi-api/storage/v1alpha1/gen/go";

// Back End (network-facing) APIs.

// NVMe/TCP and NVMe/RoCEv2

// Back End (network-facing) APIs. NVMe/TCP and NVMe/RoCEv2 protocols are covered by this service.
service NVMfRemoteControllerService {
rpc NVMfRemoteControllerConnect (NVMfRemoteControllerConnectRequest) returns (NVMfRemoteControllerConnectResponse) {}
rpc NVMfRemoteControllerDisconnect (NVMfRemoteControllerDisconnectRequest) returns (NVMfRemoteControllerDisconnectResponse) {}
Expand All @@ -33,31 +30,8 @@ message NVMfRemoteController {
bool hdgst = 7;
bool ddgst = 8;
NvmeMultipath multipath = 9;
int64 num_io_queues = 10;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://linter.aip.dev/141/count-suffix

int64 io_queues_count = 10;
int64 queue_size = 11;

}

enum NvmeTransportType {
NVME_TRANSPORT_FC = 0;
NVME_TRANSPORT_PCIE = 1;
NVME_TRANSPORT_RDMA = 2;
NVME_TRANSPORT_TCP = 3;
NVME_TRANSPORT_CUSTOM = 4;
}

enum NvmeAddressFamily {
NVMF_ADRFAM_IPV4 = 0;
NVMF_ADRFAM_IPV6 = 1;
NVMF_ADRFAM_IB = 2;
NVMF_ADRFAM_FC = 3;
NVMF_ADRFAM_INTRA_HOST = 4;
}

enum NvmeMultipath {
NVME_MPIO_DISABLE = 0;
NVME_MPIO_FAILOVER = 1;
NVME_MPIO_MULTIPATH = 2;
}

message NVMfRemoteControllerConnectRequest {
Expand Down Expand Up @@ -109,3 +83,27 @@ message NVMfRemoteControllerStatsResponse {
string stats = 2;
}

enum NvmeTransportType {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://linter.aip.dev/126/unspecified

NVME_TRANSPORT_TYPE_UNSPECIFIED = 0;
NVME_TRANSPORT_FC = 1;
NVME_TRANSPORT_PCIE = 2;
NVME_TRANSPORT_RDMA = 3;
NVME_TRANSPORT_TCP = 4;
NVME_TRANSPORT_CUSTOM = 5;
}

enum NvmeAddressFamily {
NVME_ADDRESS_FAMILY_UNSPECIFIED = 0;
NVMF_ADRFAM_IPV4 = 1;
NVMF_ADRFAM_IPV6 = 2;
NVMF_ADRFAM_IB = 3;
NVMF_ADRFAM_FC = 4;
NVMF_ADRFAM_INTRA_HOST = 5;
}

enum NvmeMultipath {
NVME_MULTIPATH_UNSPECIFIED = 0;
NVME_MULTIPATH_DISABLE = 1;
NVME_MULTIPATH_FAILOVER = 2;
NVME_MULTIPATH_MULTIPATH = 3;
}
2 changes: 1 addition & 1 deletion storage/v1alpha1/common.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ message NvmeControllerPciId {
int32 device = 2;

// Physical function, always set to 0 in current model
int32 function = 3;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://linter.aip.dev/140/reserved-words

int32 func = 3;

// SRIOV Virtual function within the Device and Physical function.
// Set to 0 for Physical Function. Virtual Function numbering starts from 1
Expand Down
12 changes: 4 additions & 8 deletions storage/v1alpha1/frontend_nvme_pcie.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,7 @@ import "uuid.proto";
import "google/protobuf/empty.proto";
import "google/api/annotations.proto";

// Front End (host-facing) APIs.

// NVMe/PCIe emulation

// Front End (host-facing) APIs. Mostly used for NVMe/PCIe emulation and host presentation.
service FrontendNvmeService {
rpc NVMeSubsystemCreate (NVMeSubsystemCreateRequest) returns (NVMeSubsystem) {
option (google.api.http) = {
Expand Down Expand Up @@ -152,7 +149,7 @@ message NVMeController {
// If not set, the xPU will provide a default.
int32 max_nsq = 5;

// maximum number of host completion queues allowed. Optional.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://linter.aip.dev/203/optional

// maximum number of host completion queues allowed.
// If not set, the xPU will provide a default.
int32 max_ncq = 6;

Expand All @@ -171,7 +168,6 @@ message NVMeController {

message NVMeNamespace {
// namespace's unique key
// replaces: int64 id = 1;
common.v1.ObjectKey id = 1;

// subsystem for this namespace
Expand All @@ -192,13 +188,13 @@ message NVMeNamespace {

// Size/Capacity of the namespace in blocks, size in bytes will
// be BlockSize x NumBlocks.
int64 num_blocks = 6;
int64 blocks_count = 6;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://linter.aip.dev/141/count-suffix


// Globally unique identifier for the namespace
string nguid = 7;

// 64bit Extended unique identifier for the namespace
// mandatory if guid is not specified, optional otherwise
// mandatory if guid is not specified
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://linter.aip.dev/203/optional

int64 eui64 = 8;

// Globally unique identifier for the namespace
Expand Down
14 changes: 5 additions & 9 deletions storage/v1alpha1/frontend_virtio_blk.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,7 @@ import "common.proto";
import "google/protobuf/empty.proto";
import "google/api/annotations.proto";

// Front End (host-facing) APIs.

// Virtio-blk emulation

// Front End (host-facing) APIs. Mostly used for Virtio-blk emulation emulation and host presentation as alternative to Nvme.
service FrontendVirtioBlkService {
rpc VirtioBlkCreate (VirtioBlkCreateRequest) returns (VirtioBlk) {
option (google.api.http) = {
Expand Down Expand Up @@ -50,11 +47,10 @@ service FrontendVirtioBlkService {

message VirtioBlk {
int64 id = 1;
string name = 2;
NvmeControllerPciId pcie_id = 3;
string bdev = 4;
int64 max_io_qps = 5;
string serial_number = 6;
NvmeControllerPciId pcie_id = 2;
string bdev = 3;
int64 max_io_qps = 4;
string serial_number = 5;
}

message VirtioBlkCreateRequest {
Expand Down
Loading