Need to handle auth_db upgrade to avoid CI test failure

[shuklanirdesh82]

Interesting excerpt from CI test failure

01/13/17 06:23:00 224047 [Thread-1] [ERROR  ] DB upgrade is not supported. Please remove the DB file at /etc/vmware/vmdkops/auth-db. All existing configuration will be removed and need to be recreated after removing the DB file.
01/13/17 06:23:00 224047 [Thread-1] [ERROR  ] Unhandled Exception:
Traceback (most recent call last):
  File "/usr/lib/vmware/vmdkops/bin/vmdk_ops.py", line 1359, in execRequestThread
    opts=opts)
  File "/usr/lib/vmware/vmdkops/bin/vmdk_ops.py", line 710, in executeRequest
    error_info, tenant_uuid, tenant_name = auth.get_tenant(vm_uuid)
  File "/usr/lib/vmware/vmdkops/Python/auth.py", line 123, in get_tenant
    _auth_mgr = get_auth_mgr()
  File "/usr/lib/vmware/vmdkops/Python/auth.py", line 48, in get_auth_mgr
    thread_local._auth_mgr.connect()
  File "/usr/lib/vmware/vmdkops/Python/auth_data.py", line 456, in connect
    raise DbConnectionError(self.db_path)
auth_data.DbConnectionError: DB connection error /etc/vmware/vmdkops/auth-db

Currently test-esx removes removes existing auth_db completely before running any tests hence the issue was not observed so far and volume creation was not complaining.

CI does resetting the testbed and it seems ESX does contain auth-db that causes the above issue.

Proposed solution: need to remove auth-db during deployment. We may enhance the check by looking at the auth-db version before removing it.

/CC @kerneltime

[kerneltime]

Ideally we should test upgrade case going forward, blindly deleting the DB during each test run seems a bit harsh. For now this should be fine but we need to track a testing issue to track how we test upgrades.

[msterin]

//CC @lipingxue

Proposed solution: need to remove auth-db during deployment. We may enhance the check by looking at the auth-db version before removing it.

Our original plan was as follows:

• introduce schema version in the DB
• force DB deletion on test to hide "no versioned" -> "versioned" schema update
• For further schema changhes, support version to version in-place upgrade

so, the auth_db deletion was a temporary measure. It should not be needed anymore, we just need to make sure we do not have really stale DBs around

Actions:

• we should drop the DB delete from test set (@liping)
• if you still have od DB, remove it manyully per error message

Comments please ?