Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

operator-sdk add api for Ansible/Helm operators. #2703

Conversation

bharathi-tenneti
Copy link
Contributor

Description: Enable Ansible/Helm operator developers to create additional APIs, through SDK CLI, using below command.
Ansible Example:
operator-sdk add api --api-version=example.com/v1alpha1 --kind=Memcached
Helm Example: (chart options being optional)
operator-sdk add api --api-version=example.com/v1alpha1 --kind=Memcached --helm-chart=stable/memcached
Motivation:
As of today, SDK CLI can be used only to add additonal APIs for Go based Operators. Ansible/Helm operator developers are not able to create additonal APIs via CLI, once the original project scaffolds. Today, developers have to manually add necessary files to the project scaffold.
For more information please refer proposal.

@openshift-ci-robot openshift-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 23, 2020
@bharathi-tenneti
Copy link
Contributor Author

To test out, please run the command in an existing project scaffold.
Please note that, There is a known issue that when adding Helm resource, deploy\role.yaml is not being updated for all permissions. Looking for inputs as well.

@bharathi-tenneti bharathi-tenneti removed the request for review from varshaprasad96 March 23, 2020 14:09
@jmrodri jmrodri changed the title <WIP> operator-sdk add api for Ansible/Helm operators. <WIP> operator-sdk add api for Ansible/Helm operators. Mar 23, 2020
@openshift-ci-robot openshift-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 25, 2020
@openshift-ci-robot openshift-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 25, 2020
@@ -309,6 +309,8 @@ kubectl delete -f deploy/role.yaml
kubectl delete -f deploy/service_account.yaml
kubectl delete -f deploy/crds/example.com_nginxes_crd.yaml
```
**NOTE** Additional CR/CRD's can be added to the project by running, for example, the command :`operator-sdk add api --api-version=cache.example.com/v1alpha1 --kind=AppService`
For more information, refer [cli][addcli] doc.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same issue faced with the watches for helm as well. See:

Screenshot 2020-04-09 at 19 17 13

The command used over the Helm Mencached sample was: $ operator-sdk add api --api-version=cache.example.com/v1alpha1 --kind=AppService

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO, this one is non-blocking. This doesn't change the behavior at all. Reordering of fields is a minor nuisance. The added watchDependentResources is a quirk of YAML marshalling due to the fact that we set watchDependentResources = true as a default when we unmarshal.

}
newWatch := watches.Watch{
GroupVersionKind: gvk,
ChartDir: HelmChartsDir + "/" + r.LowerKind,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To fix the issue @camilamacedo86 noted about the mismatched chart directory name, we need to pass chart to this function and use that to set ChartDir

Copy link
Contributor Author

@bharathi-tenneti bharathi-tenneti Apr 9, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@joelanford I am guessing this is an existing bug as well if I understand
Currently, when creating new watch, as you see below s.Resource.LowerKind is being set. Or, is it only conditional. Could get clarification here.

type WatchesYAML struct {
	input.Input

	Resource      *scaffold.Resource
	HelmChartsDir string
	ChartName     string
}

// GetInput gets the scaffold execution input
func (s *WatchesYAML) GetInput() (input.Input, error) {
	if s.Path == "" {
		s.Path = WatchesYamlFile
	}
	s.HelmChartsDir = HelmChartsDir
	s.TemplateBody = watchesYAMLTmpl
	if s.ChartName == "" {
		s.ChartName = s.Resource.LowerKind
	}
	return s.Input, nil
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's only if ChartName is empty. We set it here.

Comment on lines 149 to 150
// MergeHelmRoleForResource merges incoming new API resource rules with existing deploy/role.yaml
func MergeHelmRoleForResource(r *Resource, absProjectPath string, roleScaffold Role) error {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this specific to Helm? This can probably be renamed to MergeRoleForResource, right?

If it is specfic to Helm, we should move it to somewhere in internal/scaffold/helm.

@openshift-ci-robot openshift-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 14, 2020
@openshift-ci-robot openshift-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 14, 2020
@bharathi-tenneti
Copy link
Contributor Author

bharathi-tenneti commented Apr 15, 2020

Ansible operator testing results for add api

 $ /Users/btenneti/go/bin/operator-sdk new ansible-demo --api-version=cache.example.com/v1alpha1 --kind=Memcached --type=ansible
INFO[0000] Creating new Ansible operator 'ansible-demo'. 
INFO[0000] Created deploy/service_account.yaml          
INFO[0000] Created deploy/role.yaml                     
INFO[0000] Created deploy/role_binding.yaml             
INFO[0000] Created deploy/crds/cache.example.com_v1alpha1_memcached_cr.yaml 
INFO[0000] Created build/Dockerfile                     
INFO[0000] Created roles/memcached/README.md            
INFO[0000] Created roles/memcached/meta/main.yml        
INFO[0000] Created roles/memcached/files/.placeholder   
INFO[0000] Created roles/memcached/templates/.placeholder 
INFO[0000] Created roles/memcached/vars/main.yml        
INFO[0000] Created molecule/test-local/converge.yml     
INFO[0000] Created roles/memcached/defaults/main.yml    
INFO[0000] Created roles/memcached/tasks/main.yml       
INFO[0000] Created molecule/default/molecule.yml        
INFO[0000] Created molecule/default/prepare.yml         
INFO[0000] Created molecule/default/converge.yml        
INFO[0000] Created molecule/default/verify.yml          
INFO[0000] Created roles/memcached/handlers/main.yml    
INFO[0000] Created watches.yaml                         
INFO[0000] Created deploy/operator.yaml                 
INFO[0000] Created .travis.yml                          
INFO[0000] Created requirements.yml                     
INFO[0000] Created molecule/test-local/molecule.yml     
INFO[0000] Created molecule/test-local/prepare.yml      
INFO[0000] Created molecule/test-local/verify.yml       
INFO[0000] Created molecule/cluster/molecule.yml        
INFO[0000] Created molecule/cluster/create.yml          
INFO[0000] Created molecule/cluster/prepare.yml         
INFO[0000] Created molecule/cluster/converge.yml        
INFO[0000] Created molecule/cluster/verify.yml          
INFO[0000] Created molecule/cluster/destroy.yml         
INFO[0000] Created molecule/templates/operator.yaml.j2  
INFO[0000] Generated CustomResourceDefinition manifests. 
INFO[0000] Project creation complete.   
 $ cd ansible-demo/
 $ /Users/btenneti/go/bin/operator-sdk add api  --api-version=app.example.com/v1alpha1 --kind=Mykind
INFO[0000] Generating api version app.example.com/v1alpha1 for kind Mykind. 
INFO[0000] Created deploy/crds/app.example.com_v1alpha1_mykind_cr.yaml 
INFO[0000] Created roles/mykind/README.md               
INFO[0000] Created roles/mykind/meta/main.yml           
INFO[0000] Created roles/mykind/files/.placeholder      
INFO[0000] Created roles/mykind/templates/.placeholder  
INFO[0000] Created roles/mykind/vars/main.yml           
INFO[0000] Created roles/mykind/defaults/main.yml       
INFO[0000] Created roles/mykind/tasks/main.yml          
INFO[0000] Created roles/mykind/handlers/main.yml       
INFO[0000] Generated CustomResourceDefinition manifests. 
INFO[0000] API generation complete.         
$ cat watches.yaml 
---
- version: v1alpha1
  group: cache.example.com
  kind: Memcached
  role: memcached
- version: v1alpha1
  group: app.example.com
  kind: Mykind
  role: mykind
$ tree roles
roles
├── memcached
│   ├── README.md
│   ├── defaults
│   │   └── main.yml
│   ├── files
│   ├── handlers
│   │   └── main.yml
│   ├── meta
│   │   └── main.yml
│   ├── tasks
│   │   └── main.yml
│   ├── templates
│   └── vars
│       └── main.yml
└── mykind
    ├── README.md
    ├── defaults
    │   └── main.yml
    ├── files
    ├── handlers
    │   └── main.yml
    ├── meta
    │   └── main.yml
    ├── tasks
    │   └── main.yml
    ├── templates
    └── vars
        └── main.yml
$ tree deploy/crds/
deploy/crds/
├── app.example.com_mykinds_crd.yaml
├── app.example.com_v1alpha1_mykind_cr.yaml
├── cache.example.com_memcacheds_crd.yaml
└── cache.example.com_v1alpha1_memcached_cr.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  creationTimestamp: null
  name: ansible-demo
rules:
- apiGroups:
  - ""
  resources:
  - pods
  - services
  - services/finalizers
  - endpoints
  - persistentvolumeclaims
  - events
  - configmaps
  - secrets
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - apps
  resources:
  - deployments
  - daemonsets
  - replicasets
  - statefulsets
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - monitoring.coreos.com
  resources:
  - servicemonitors
  verbs:
  - get
  - create
- apiGroups:
  - apps
  resourceNames:
  - ansible-demo
  resources:
  - deployments/finalizers
  verbs:
  - update
- apiGroups:
  - ""
  resources:
  - pods
  verbs:
  - get
- apiGroups:
  - apps
  resources:
  - replicasets
  - deployments
  verbs:
  - get
- apiGroups:
  - cache.example.com
  resources:
  - '*'
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - app.example.com
  resources:
  - '*'
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
$ cat roles/memcached/tasks/main.yml 
- name: Create ConfigMap to test ansible api
  k8s:
    definition:
      kind: ConfigMap
      apiVersion: v1
      metadata:
        name: example-memcached
        namespace: default
      data:
        name: bharathi
$ cat roles/mykind/tasks/main.yml 
- name: Create ConfigMap to test ansible api
  k8s:
    definition:
      kind: ConfigMap
      apiVersion: v1
      metadata:
        name: example-mykind
        namespace: default
      data:
        name: bharathi
$ kubectl get all
NAME                                READY   STATUS    RESTARTS   AGE
pod/ansible-demo-64bd69d955-wc6s2   1/1     Running   0          87s
NAME                           TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)             AGE
service/ansible-demo-metrics   ClusterIP   10.96.1.135   <none>        8383/TCP,8686/TCP   63s
service/kubernetes             ClusterIP   10.96.0.1     <none>        443/TCP             8m51s
NAME                           READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/ansible-demo   1/1     1            1           87s
NAME                                      DESIRED   CURRENT   READY   AGE
replicaset.apps/ansible-demo-64bd69d955   1         1         1       87s

$ kubectl get configmaps
NAME                DATA   AGE
ansible-demo-lock   0      69s
example-memcached   1      45s
example-mykind      1      32s

@bharathi-tenneti
Copy link
Contributor Author

bharathi-tenneti commented Apr 15, 2020

Helm Operator testing results for add api.

$ /Users/btenneti/go/bin/operator-sdk new helm-demo --helm-chart=stable/nginx-ingress --type=helm
INFO[0000] Creating new Helm operator 'helm-demo'.      
INFO[0001] Created helm-charts/nginx-ingress            
INFO[0001] Generating RBAC rules                        
INFO[0001] Scaffolding ClusterRole and ClusterRolebinding for cluster scoped resources in the helm chart 
WARN[0001] The RBAC rules generated in deploy/role.yaml are based on the chart's default manifest. Some rules may be missing for resources that are only enabled with custom values, and some existing rules may be overly broad. Double check the rules generated in deploy/role.yaml to ensure they meet the operator's permission requirements. 
INFO[0001] Created build/Dockerfile                     
INFO[0001] Created watches.yaml                         
INFO[0001] Created deploy/service_account.yaml          
INFO[0001] Created deploy/role.yaml                     
INFO[0001] Created deploy/role_binding.yaml             
INFO[0001] Created deploy/operator.yaml                 
INFO[0001] Created deploy/crds/charts.helm.k8s.io_v1alpha1_nginxingress_cr.yaml 
INFO[0001] Generated CustomResourceDefinition manifests. 
INFO[0001] Project creation complete.  
 $ cd helm-demo/
 $ /Users/btenneti/go/bin/operator-sdk add api --api-version=cache.example.com/v1alpha1 --kind=Memcached
INFO[0000] Generating api version cache.example.com/v1alpha1 for kind Memcached. 
INFO[0000] Created helm-charts/memcached                
INFO[0000] Created deploy/crds/cache.example.com_v1alpha1_memcached_cr.yaml 
INFO[0000] Generated CustomResourceDefinition manifests. 
INFO[0000] Generating RBAC rules                        
WARN[0000] The RBAC rules generated in deploy/role.yaml are based on the chart's default manifest. Some rules may be missing for resources that are only enabled with custom values, and some existing rules may be overly broad. Double check the rules generated in deploy/role.yaml to ensure they meet the operator's permission requirements. 
INFO[0000] API generation complete.                     
 $ cat watches.yaml 
- group: charts.helm.k8s.io
  version: v1alpha1
  kind: NginxIngress
  chart: helm-charts/nginx-ingress
  watchDependentResources: true
- group: cache.example.com
  version: v1alpha1
  kind: Memcached
  chart: helm-charts/memcached
$ tree helm-charts/
helm-charts/
├── memcached
│   ├── Chart.yaml
│   ├── charts
│   ├── templates
│   │   ├── NOTES.txt
│   │   ├── _helpers.tpl
│   │   ├── deployment.yaml
│   │   ├── ingress.yaml
│   │   ├── service.yaml
│   │   ├── serviceaccount.yaml
│   │   └── tests
│   │       └── test-connection.yaml
│   └── values.yaml
└── nginx-ingress
    ├── Chart.yaml
    ├── OWNERS
    ├── README.md
    ├── ci
    │   ├── daemonset-customconfig-values.yaml
    │   ├── daemonset-customnodeport-values.yaml
    │   ├── daemonset-headers-values.yaml
    │   ├── daemonset-nodeport-values.yaml
    │   ├── daemonset-tcp-udp-configMapNamespace-values.yaml
    │   ├── daemonset-tcp-udp-values.yaml
    │   ├── daemonset-tcp-values.yaml
    │   ├── deamonset-default-values.yaml
    │   ├── deamonset-metrics-values.yaml
    │   ├── deamonset-psp-values.yaml
    │   ├── deamonset-webhook-and-psp-values.yaml
    │   ├── deamonset-webhook-values.yaml
    │   ├── deployment-autoscaling-values.yaml
    │   ├── deployment-customconfig-values.yaml
    │   ├── deployment-customnodeport-values.yaml
    │   ├── deployment-default-values.yaml
    │   ├── deployment-headers-values.yaml
    │   ├── deployment-metrics-values.yaml
    │   ├── deployment-nodeport-values.yaml
    │   ├── deployment-psp-values.yaml
    │   ├── deployment-tcp-udp-configMapNamespace-values.yaml
    │   ├── deployment-tcp-udp-values.yaml
    │   ├── deployment-tcp-values.yaml
    │   ├── deployment-webhook-and-psp-values.yaml
    │   └── deployment-webhook-values.yaml
    ├── templates
    │   ├── NOTES.txt
    │   ├── _helpers.tpl
    │   ├── addheaders-configmap.yaml
    │   ├── admission-webhooks
    │   │   ├── job-patch
    │   │   │   ├── clusterrole.yaml
    │   │   │   ├── clusterrolebinding.yaml
    │   │   │   ├── job-createSecret.yaml
    │   │   │   ├── job-patchWebhook.yaml
    │   │   │   ├── psp.yaml
    │   │   │   ├── role.yaml
    │   │   │   ├── rolebinding.yaml
    │   │   │   └── serviceaccount.yaml
    │   │   └── validating-webhook.yaml
    │   ├── clusterrole.yaml
    │   ├── clusterrolebinding.yaml
    │   ├── controller-configmap.yaml
    │   ├── controller-daemonset.yaml
    │   ├── controller-deployment.yaml
    │   ├── controller-hpa.yaml
    │   ├── controller-metrics-service.yaml
    │   ├── controller-poddisruptionbudget.yaml
    │   ├── controller-prometheusrules.yaml
    │   ├── controller-psp.yaml
    │   ├── controller-role.yaml
    │   ├── controller-rolebinding.yaml
    │   ├── controller-service.yaml
    │   ├── controller-serviceaccount.yaml
    │   ├── controller-servicemonitor.yaml
    │   ├── controller-webhook-service.yaml
    │   ├── default-backend-deployment.yaml
    │   ├── default-backend-poddisruptionbudget.yaml
    │   ├── default-backend-psp.yaml
    │   ├── default-backend-role.yaml
    │   ├── default-backend-rolebinding.yaml
    │   ├── default-backend-service.yaml
    │   ├── default-backend-serviceaccount.yaml
    │   ├── proxyheaders-configmap.yaml
    │   ├── tcp-configmap.yaml
    │   └── udp-configmap.yaml
    └── values.yaml

$ tree deploy/crds/
deploy/crds/
├── cache.example.com_memcacheds_crd.yaml
├── cache.example.com_v1alpha1_memcached_cr.yaml
├── charts.helm.k8s.io_nginxingresses_crd.yaml
└── charts.helm.k8s.io_v1alpha1_nginxingress_cr.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  creationTimestamp: null
  name: helm-demo
rules:
- apiGroups:
  - ""
  resources:
  - namespaces
  verbs:
  - get
- apiGroups:
  - ""
  resources:
  - configmaps
  - secrets
  - serviceaccounts
  - services
  verbs:
  - '*'
- apiGroups:
  - ""
  resources:
  - events
  verbs:
  - create
- apiGroups:
  - rbac.authorization.k8s.io
  resources:
  - clusterrolebindings
  - clusterroles
  verbs:
  - '*'
- apiGroups:
  - apps
  resources:
  - deployments
  verbs:
  - '*'
- apiGroups:
  - ""
  resources:
  - serviceaccounts
  - services
  verbs:
  - '*'
- apiGroups:
  - rbac.authorization.k8s.io
  resources:
  - rolebindings
  - roles
  verbs:
  - '*'
- apiGroups:
  - monitoring.coreos.com
  resources:
  - servicemonitors
  verbs:
  - get
  - create
- apiGroups:
  - apps
  resourceNames:
  - helm-demo
  resources:
  - deployments/finalizers
  verbs:
  - update
- apiGroups:
  - ""
  resources:
  - pods
  verbs:
  - get
- apiGroups:
  - apps
  resources:
  - replicasets
  - deployments
  verbs:
  - get
- apiGroups:
  - charts.helm.k8s.io
  resources:
  - '*'
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
- apiGroups:
  - cache.example.com
  resources:
  - '*'
  verbs:
  - create
  - delete
  - get
  - list
  - patch
  - update
  - watch
$ kubectl get all
NAME                                                                  READY   STATUS    RESTARTS   AGE
pod/example-memcached-68d84ffc74-f4hz8                                1/1     Running   0          22s
pod/example-nginxingress-nginx-ingress-controller-7bc57b7b58-lsx77    1/1     Running   0          53s
pod/example-nginxingress-nginx-ingress-default-backend-8cbdb6dkxc58   1/1     Running   0          53s
pod/helm-demo-6d89ddf4dc-8rx7p                                        1/1     Running   0          88s

NAME                                                         TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
service/example-memcached                                    ClusterIP      10.96.77.46     <none>        80/TCP                       22s
service/example-nginxingress-nginx-ingress-controller        LoadBalancer   10.96.123.158   <pending>     80:31534/TCP,443:30113/TCP   53s
service/example-nginxingress-nginx-ingress-default-backend   ClusterIP      10.96.208.10    <none>        80/TCP                       53s
service/helm-demo-metrics                                    ClusterIP      10.96.148.238   <none>        8383/TCP,8686/TCP            78s
service/kubernetes                                           ClusterIP      10.96.0.1       <none>        443/TCP                      9m55s

NAME                                                                 READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/example-memcached                                    1/1     1            1           22s
deployment.apps/example-nginxingress-nginx-ingress-controller        1/1     1            1           53s
deployment.apps/example-nginxingress-nginx-ingress-default-backend   1/1     1            1           53s
deployment.apps/helm-demo                                            1/1     1            1           88s

NAME                                                                           DESIRED   CURRENT   READY   AGE
replicaset.apps/example-memcached-68d84ffc74                                   1         1         1       22s
replicaset.apps/example-nginxingress-nginx-ingress-controller-7bc57b7b58       1         1         1       53s
replicaset.apps/example-nginxingress-nginx-ingress-default-backend-8cbdb6d68   1         1         1       53s
replicaset.apps/helm-demo-6d89ddf4dc                                           1         1         1       88s

estroz
estroz previously requested changes Apr 16, 2020
Copy link
Member

@estroz estroz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A bunch of questions and nits.

@bharathi-tenneti bharathi-tenneti force-pushed the ansible/helm-addapi branch 2 times, most recently from 392297a to dbe0a82 Compare April 16, 2020 15:51
@bharathi-tenneti bharathi-tenneti requested a review from estroz April 16, 2020 17:57
@bharathi-tenneti
Copy link
Contributor Author

/unhold

@openshift-ci-robot openshift-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Apr 17, 2020
@jmrodri jmrodri dismissed estroz’s stale review April 17, 2020 14:35

@estroz left comments left 2 days ago. Bharathi addressed them all. Dismissing the review in order to merge.

@bharathi-tenneti bharathi-tenneti merged commit 1c1386f into operator-framework:master Apr 17, 2020
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants