02/10/17 12:24:08 190940 [MainThread] [INFO   ] Log configuration generated - '/etc/vmware/vmdkops/log_config.json'.
02/10/17 12:24:08 190940 [MainThread] [INFO   ] === Starting vmdkops service ====
02/10/17 12:24:08 190940 [MainThread] [INFO   ] Loading VMCI server lib.
02/10/17 12:24:08 190940 [MainThread] [INFO   ] Connecting to the local Service Instance as 'dcui' 
02/14/17 07:45:51 190940 [Thread-1] [INFO   ] Connecting to the local Service Instance as 'dcui' 
02/14/17 07:45:51 190940 [Thread-1] [INFO   ] Created /vmfs/volumes/msa-2012i-prod-0/dockvols
02/14/17 07:45:52 190940 [Thread-1] [INFO   ] Created /vmfs/volumes/msa-2012i-store-0/dockvols
02/14/17 07:45:52 190940 [Thread-1] [INFO   ] Created /vmfs/volumes/msa-2012i-store-1/dockvols
02/14/17 07:45:52 190940 [Thread-1] [INFO   ] Created /vmfs/volumes/msa-2312i-store-0/dockvols
02/14/17 07:45:53 190940 [Thread-1] [INFO   ] Created /vmfs/volumes/msa-2312i-prod-0/dockvols
02/14/17 07:45:53 190940 [Thread-1] [INFO   ] Created /vmfs/volumes/msa-2312i-store-1/dockvols
02/14/17 07:45:53 190940 [Thread-1] [INFO   ] Created /vmfs/volumes/msa-2312i-prod-1/dockvols
02/14/17 07:45:53 190940 [Thread-1] [INFO   ] Created /vmfs/volumes/msa-2312i-docker-store-0/dockvols
02/14/17 07:45:54 190940 [Thread-1] [INFO   ] Created /vmfs/volumes/msa-2312i-store-2/dockvols
02/14/17 07:45:54 190940 [docker-node-2-nolock-list] [INFO   ] executeRequest 'list' completed with ret=[]
02/14/17 08:15:25 190940 [Thread-2] [INFO   ] cmd get with opts {} on tenant_uuid 16ab69cb-ec3b-4943-b71f-f3cb296f53c9 datastore msa-2312i-store-0 is allowed to execute
02/14/17 08:15:25 190940 [Thread-2] [INFO   ] Symlink /vmfs/volumes/msa-2312i-store-0/dockvols/_DEFAULT is created to point to path /vmfs/volumes/msa-2312i-store-0/dockvols/16ab69cb-ec3b-4943-b71f-f3cb296f53c9
02/14/17 08:15:25 190940 [Thread-2] [INFO   ] Created /vmfs/volumes/msa-2312i-store-0/dockvols/16ab69cb-ec3b-4943-b71f-f3cb296f53c9
02/14/17 08:15:25 190940 [docker-node-2-[msa-2312i-store-0] dockvols/16ab69cb-ec3b-4943-b71f-f3cb296f53c9/f52c24d3a7ec9a507457268e2e059917b2fa1cfa65bfe575223ca23993576631.vmdk] [INFO   ] executeRequest 'get' completed with ret={'Error': 'Volume f52c24d3a7ec9a507457268e2e059917b2fa1cfa65bfe575223ca23993576631 not found (file: /vmfs/volumes/msa-2312i-store-0/dockvols/16ab69cb-ec3b-4943-b71f-f3cb296f53c9/f52c24d3a7ec9a507457268e2e059917b2fa1cfa65bfe575223ca23993576631.vmdk)'}
02/14/17 09:16:38 190940 [docker-node-2-nolock-list] [INFO   ] executeRequest 'list' completed with ret=[]
02/14/17 13:45:19 190940 [Thread-4] [WARNING] Reconnecting to the local Service Instance
02/14/17 13:45:19 190940 [Thread-4] [INFO   ] cmd get with opts {} on tenant_uuid 16ab69cb-ec3b-4943-b71f-f3cb296f53c9 datastore msa-2312i-store-0 is allowed to execute
02/14/17 13:45:19 190940 [docker-node-2-[msa-2312i-store-0] dockvols/16ab69cb-ec3b-4943-b71f-f3cb296f53c9/2dd63a11254b9313e917ff612df76f4e4ee48849a2ae8570fae02eb18fbaec22.vmdk] [INFO   ] executeRequest 'get' completed with ret={'Error': 'Volume 2dd63a11254b9313e917ff612df76f4e4ee48849a2ae8570fae02eb18fbaec22 not found (file: /vmfs/volumes/msa-2312i-store-0/dockvols/16ab69cb-ec3b-4943-b71f-f3cb296f53c9/2dd63a11254b9313e917ff612df76f4e4ee48849a2ae8570fae02eb18fbaec22.vmdk)'}
02/14/17 13:45:41 190940 [docker-node-2-nolock-list] [INFO   ] executeRequest 'list' completed with ret=[]
02/14/17 13:45:51 190940 [docker-node-2-nolock-list] [INFO   ] executeRequest 'list' completed with ret=[]
02/14/17 13:45:53 190940 [docker-node-2-nolock-list] [INFO   ] executeRequest 'list' completed with ret=[]
02/14/17 13:45:54 190940 [docker-node-2-nolock-list] [INFO   ] executeRequest 'list' completed with ret=[]
02/14/17 13:45:55 190940 [docker-node-2-nolock-list] [INFO   ] executeRequest 'list' completed with ret=[]
02/14/17 13:52:17 190940 [docker-node-2-nolock-list] [INFO   ] executeRequest 'list' completed with ret=[]
02/14/17 13:59:31 190940 [docker-node-2-nolock-list] [INFO   ] executeRequest 'list' completed with ret=[]
02/14/17 13:59:33 190940 [docker-node-2-nolock-list] [INFO   ] executeRequest 'list' completed with ret=[]
02/15/17 08:17:21 190940 [docker-node-2-nolock-list] [INFO   ] executeRequest 'list' completed with ret=[]
02/15/17 10:15:11 190940 [docker-node-2-nolock-list] [INFO   ] executeRequest 'list' completed with ret=[]
02/15/17 10:15:14 190940 [docker-node-2-nolock-list] [INFO   ] executeRequest 'list' completed with ret=[]
02/15/17 10:17:44 267199 [MainThread] [INFO   ] Connecting to the local Service Instance as 'dcui' 
02/15/17 10:18:32 267209 [MainThread] [INFO   ] Connecting to the local Service Instance as 'dcui' 
02/16/17 07:36:49 190940 [Thread-16] [WARNING] Reconnecting to the local Service Instance
02/16/17 07:36:49 190940 [Thread-16] [ERROR  ] Unhandled Exception:
Traceback (most recent call last):
  File "/usr/lib/vmware/vmdkops/bin/vmdk_ops.py", line 1373, in execRequestThread
    opts=opts)
  File "/usr/lib/vmware/vmdkops/bin/vmdk_ops.py", line 746, in executeRequest
    % (datastore, ", ".join(get_datastore_names_list), vm_datastore))
TypeError: can only join an iterable
02/16/17 07:36:50 190940 [Thread-17] [ERROR  ] Unhandled Exception:
Traceback (most recent call last):
  File "/usr/lib/vmware/vmdkops/bin/vmdk_ops.py", line 1373, in execRequestThread
    opts=opts)
  File "/usr/lib/vmware/vmdkops/bin/vmdk_ops.py", line 746, in executeRequest
    % (datastore, ", ".join(get_datastore_names_list), vm_datastore))
TypeError: can only join an iterable
02/16/17 08:24:45 190940 [Thread-18] [WARNING] Reconnecting to the local Service Instance
02/16/17 08:24:45 190940 [Thread-18] [INFO   ] cmd get with opts {} on tenant_uuid 16ab69cb-ec3b-4943-b71f-f3cb296f53c9 datastore msa-2312i-docker-store-0 is allowed to execute
02/16/17 08:24:45 190940 [Thread-18] [INFO   ] Created /vmfs/volumes/msa-2312i-docker-store-0/dockvols/16ab69cb-ec3b-4943-b71f-f3cb296f53c9
02/16/17 08:24:45 190940 [docker-node-2-[msa-2312i-docker-store-0] dockvols/16ab69cb-ec3b-4943-b71f-f3cb296f53c9/Test.vmdk] [INFO   ] executeRequest 'get' completed with ret={'Error': 'Volume Test not found (file: /vmfs/volumes/msa-2312i-docker-store-0/dockvols/16ab69cb-ec3b-4943-b71f-f3cb296f53c9/Test.vmdk)'}
02/16/17 08:24:45 190940 [Thread-19] [INFO   ] cmd create with opts {'size': '10GB', 'fstype': 'ext4', 'type': 'thin'} on tenant_uuid 16ab69cb-ec3b-4943-b71f-f3cb296f53c9 datastore msa-2312i-docker-store-0 is allowed to execute
02/16/17 08:24:45 190940 [docker-node-2-[msa-2312i-docker-store-0] dockvols/16ab69cb-ec3b-4943-b71f-f3cb296f53c9/Test.vmdk] [INFO   ] *** createVMDK: /vmfs/volumes/msa-2312i-docker-store-0/dockvols/16ab69cb-ec3b-4943-b71f-f3cb296f53c9/Test.vmdk opts = {'size': '10GB', 'fstype': 'ext4', 'type': 'thin'}
02/16/17 08:24:45 190940 [docker-node-2-[msa-2312i-docker-store-0] dockvols/16ab69cb-ec3b-4943-b71f-f3cb296f53c9/Test.vmdk] [INFO   ] executeRequest 'create' completed with ret={'Error': "Invalid options: ['type'] \nValid options and defaults: <zip object at 0xe848a22588>"}
02/16/17 08:25:30 190940 [Thread-20] [INFO   ] cmd get with opts {} on tenant_uuid 16ab69cb-ec3b-4943-b71f-f3cb296f53c9 datastore msa-2312i-docker-store-0 is allowed to execute
02/16/17 08:25:30 190940 [docker-node-2-[msa-2312i-docker-store-0] dockvols/16ab69cb-ec3b-4943-b71f-f3cb296f53c9/Test.vmdk] [INFO   ] executeRequest 'get' completed with ret={'Error': 'Volume Test not found (file: /vmfs/volumes/msa-2312i-docker-store-0/dockvols/16ab69cb-ec3b-4943-b71f-f3cb296f53c9/Test.vmdk)'}
02/16/17 08:25:30 190940 [Thread-21] [INFO   ] cmd create with opts {'size': '10gb', 'diskformat': 'thin', 'fstype': 'ext4'} on tenant_uuid 16ab69cb-ec3b-4943-b71f-f3cb296f53c9 datastore msa-2312i-docker-store-0 is allowed to execute
02/16/17 08:25:30 190940 [docker-node-2-[msa-2312i-docker-store-0] dockvols/16ab69cb-ec3b-4943-b71f-f3cb296f53c9/Test.vmdk] [INFO   ] *** createVMDK: /vmfs/volumes/msa-2312i-docker-store-0/dockvols/16ab69cb-ec3b-4943-b71f-f3cb296f53c9/Test.vmdk opts = {'size': '10gb', 'diskformat': 'thin', 'fstype': 'ext4'}
02/16/17 08:25:30 190940 [docker-node-2-[msa-2312i-docker-store-0] dockvols/16ab69cb-ec3b-4943-b71f-f3cb296f53c9/Test.vmdk] [WARNING] Reconnecting to the local Service Instance
02/16/17 08:25:31 190940 [docker-node-2-[msa-2312i-docker-store-0] dockvols/16ab69cb-ec3b-4943-b71f-f3cb296f53c9/Test.vmdk] [INFO   ] executeRequest 'create' completed with ret=None
02/16/17 08:25:31 190940 [Thread-22] [INFO   ] cmd attach with opts {} on tenant_uuid 16ab69cb-ec3b-4943-b71f-f3cb296f53c9 datastore msa-2312i-docker-store-0 is allowed to execute
02/16/17 08:25:31 190940 [docker-node-2-[msa-2312i-docker-store-0] dockvols/16ab69cb-ec3b-4943-b71f-f3cb296f53c9/Test.vmdk] [INFO   ] *** attachVMDK: /vmfs/volumes/msa-2312i-docker-store-0/dockvols/16ab69cb-ec3b-4943-b71f-f3cb296f53c9/Test.vmdk to docker-node-2 VM uuid = 422dacfb-6e4a-a4de-2798-350826a9fc6c
02/16/17 08:25:31 190940 [docker-node-2-[msa-2312i-docker-store-0] dockvols/16ab69cb-ec3b-4943-b71f-f3cb296f53c9/Test.vmdk] [INFO   ] Attaching /vmfs/volumes/msa-2312i-docker-store-0/dockvols/16ab69cb-ec3b-4943-b71f-f3cb296f53c9/Test.vmdk as independent_persistent
02/16/17 08:25:31 190940 [docker-node-2-[msa-2312i-docker-store-0] dockvols/16ab69cb-ec3b-4943-b71f-f3cb296f53c9/Test.vmdk] [INFO   ] Disk /vmfs/volumes/msa-2312i-docker-store-0/dockvols/16ab69cb-ec3b-4943-b71f-f3cb296f53c9/Test.vmdk successfully attached. controller pci_slot_number=160, disk_slot=1
02/16/17 08:25:31 190940 [docker-node-2-[msa-2312i-docker-store-0] dockvols/16ab69cb-ec3b-4943-b71f-f3cb296f53c9/Test.vmdk] [INFO   ] executeRequest 'attach' completed with ret={'Unit': '1', 'ControllerPciSlotNumber': '160'}
02/16/17 08:25:35 190940 [Thread-23] [INFO   ] cmd detach with opts {} on tenant_uuid 16ab69cb-ec3b-4943-b71f-f3cb296f53c9 datastore msa-2312i-docker-store-0 is allowed to execute
02/16/17 08:25:35 190940 [docker-node-2-[msa-2312i-docker-store-0] dockvols/16ab69cb-ec3b-4943-b71f-f3cb296f53c9/Test.vmdk] [INFO   ] *** detachVMDK: /vmfs/volumes/msa-2312i-docker-store-0/dockvols/16ab69cb-ec3b-4943-b71f-f3cb296f53c9/Test.vmdk from docker-node-2 VM uuid = 422dacfb-6e4a-a4de-2798-350826a9fc6c
02/16/17 08:25:36 190940 [docker-node-2-[msa-2312i-docker-store-0] dockvols/16ab69cb-ec3b-4943-b71f-f3cb296f53c9/Test.vmdk] [INFO   ] Disk detached /vmfs/volumes/msa-2312i-docker-store-0/dockvols/16ab69cb-ec3b-4943-b71f-f3cb296f53c9/Test.vmdk
02/16/17 08:25:36 190940 [docker-node-2-[msa-2312i-docker-store-0] dockvols/16ab69cb-ec3b-4943-b71f-f3cb296f53c9/Test.vmdk] [INFO   ] executeRequest 'detach' completed with ret=None
02/16/17 08:25:42 190940 [docker-node-2-nolock-list] [INFO   ] executeRequest 'list' completed with ret=[{'Attributes': {}, 'Name': 'Test@msa-2312i-docker-store-0'}]