Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

chef-solo launched by vagrant on Windows guest throws Chef::Exceptions::MissingRole #4974

Closed
lonniev opened this issue Dec 15, 2014 · 57 comments

Comments

@lonniev
Copy link
Contributor

lonniev commented Dec 15, 2014

Similiar to (closed) #2818

==> vb-tt-sync: [2014-12-14T18:49:44-08:00] INFO: Setting the run_list to ["role[tasktop-sync-server]"] from CLI options
==> vb-tt-sync: [2014-12-14T18:49:44-08:00] DEBUG: Applying attributes from json file
==> vb-tt-sync: [2014-12-14T18:49:44-08:00] DEBUG: Platform is windows version 6.1.7601
==> vb-tt-sync: [2014-12-14T18:49:50-08:00] ERROR: Role tasktop-sync-server (included by 'top level') is in the runlist but does not exist. Skipping expand.
==> vb-tt-sync: 
==> vb-tt-sync: 
==> vb-tt-sync: ================================================================================
==> vb-tt-sync: Error expanding the run_list:
==> vb-tt-sync: ================================================================================
==> vb-tt-sync: 
==> vb-tt-sync: Missing Role(s) in Run List:
==> vb-tt-sync: ----------------------------
==> vb-tt-sync: * tasktop-sync-server included by 'top level'
==> vb-tt-sync: 
==> vb-tt-sync: Original Run List
==> vb-tt-sync: -----------------
==> vb-tt-sync: * role[tasktop-sync-server]
==> vb-tt-sync: 
==> vb-tt-sync: [2014-12-14T18:49:50-08:00] DEBUG: Re-raising exception: Chef::Exceptions::MissingRole - The expanded run list includes nonexistent roles: tasktop-sync-server

The Vagrantfile includes:

    vmh.vm.provision "chef_solo" do |chef|

      chef.log_level = :debug

      chef.cookbooks_path = "./cookbooks"
      chef.roles_path = "./roles"
      chef.data_bags_path = "./bags"

      chef.add_role "tasktop-sync-server"
    end

The entire configuration works fine for Ubuntu guest boxes. The provider is virtualbox. All the proper files are in the right directories on the host, all the right mounts of shared folders are in place, all the remapped files are available on the guest Windows, all the files can be opened from the Windows guest file explorer, and the solo.rb file has the proper, mapped paths for cookbooks, roles, and bags. If chef.add_role is replaced with chef.add_recipe for a recipe that is in the cookbooks folder, then that works fine. It is just the role access that is failing.

So, what could be the problem?

(Is there a unix-dos CRLF issue at the root of this?)

@sethvargo
Copy link
Contributor

Hey @lonniev,

Could you please send us the complete Vagrantfile and the contents of the Vagrant command running in debug mode. Because this output is rather lengthy, please use a service like GitHub Gist, Pastebin, or send the logs as an email attachment.

@sneal have you seen this with Windows before?

@lonniev
Copy link
Contributor Author

lonniev commented Dec 15, 2014

https://gist.github.com/lonniev/8ab5dacf10cb877cf717

That's just the Vagrantfile. The VAGRANT_LOG debug is vast. Anything you want to focus on? Naively, I observe it is doing a lot of the right expected stuff and then, at the end, without a lot of helpful context, it makes the same complaint about the empty run list and unwinds the uncaught exception. I can capture the whole log of course but if you want me to grep out something, I could save an electronic forest with a little filtering first.

@sethvargo
Copy link
Contributor

@lonniev can you just gist the entire debug log please? It's expected to be really long 😄

@lonniev
Copy link
Contributor Author

lonniev commented Dec 15, 2014

@lonniev
Copy link
Contributor Author

lonniev commented Dec 15, 2014

And here's the chef-solo-2 roles folder on the guest
https://www.dropbox.com/s/0mllufhetu8tnwc/Screenshot%202014-12-14%2022.18.47.png?dl=0

@lonniev
Copy link
Contributor Author

lonniev commented Dec 15, 2014

name "tasktop-sync-server"
description "Role for a Server running Tasktop Sync Studio"

run_list(
        "recipe[tasktop-sync-studio]"
        )

I'll debug more tomorrow.

@MartinSGill
Copy link

I've been seeing this as well. As I'm fairly new to vagrant and chef, I thought it was something I was doing.

  config.vm.define "win" do |win|
    win.vm.provision "chef_solo" do |chef|
      # Want to use smb for windows, but there's a bug in 1.6.5
      # that stops it working. So virtualbox for now
      chef.synced_folder_type = :virtualbox
      chef.cookbooks_path = "./chef-library"
      chef.roles_path = "./chef-data/roles"
      chef.data_bags_path = "./chef-data/data_bags"
      chef.add_role "dev"
    end
  end
{
  "name": "dev",
  "description": "Role for development. Edit as required.",
  "json_class": "Chef::Role",
  "default_attributes": {
  },
  "override_attributes": {
    "chef_client": { }
  },
  "chef_type": "role",
  "run_list": [
      "notepadplusplus"
  ],
  "env_run_lists": {
  }
}

I have an identical config for a linux machine, and that works as expected.

@sethvargo
Copy link
Contributor

Hunch - what if you specify the full path to the roles (instead of relative)?

@MartinSGill
Copy link

I'll try, but currently I'm blocked by #4976.

@MartinSGill
Copy link

@sethvargo Tried your suggestion. Made no difference.

edit:

Also tried chef.synced_folder_type = :smb but that didn't help either.

@MartinSGill
Copy link

Some more info in this gist, where I ran the guest scripts directly.

The path and seems to exist. How exactly does vagrant tell chef-solo where to look for the roles, i.e. where the paths are?

@lonniev
Copy link
Contributor Author

lonniev commented Dec 15, 2014

Actually, looking at your gist, I see that you deduced all what I say below. Your command-line call is exactly what the elevated powershell script calls when vagrant commands the guest to run chef-solo.

@MartinSGill config-solo, for a vagrant use case, seems to be configured with a solo.rb and a dna.json. The former file provides configuration settings such as the paths to cookbooks, data bags, environments, and roles while the latter (in my instances) provides the run list of recipes and roles.

These two files are found on a Windows guest at c:\tmp\vagrant-chef-2.

The contents of these files comes from statements in the chef provisioner block within the Vagrantfile.

Beyond this reply, I defer to the Vagrant and Opscode folk who know more and can correct my flawed assumptions.

@lonniev
Copy link
Contributor Author

lonniev commented Dec 15, 2014

@sethvargo when stating the location of the paths within the chef provisioner block of the Vagrantfile, my assumption is that the pathnames should reference (relatively or absolutely) the directories as they are located on the Host OS. Then, it is vagrant and the chosen provisioner that take care of constructing mount points within the guest OS and mapping the external Host directories as stated to the internal Guest mount points (or physically importing and moving files to achieve the same copied effect).

Given that, how could it matter if the statements in the Vagrantfile used relative versus absolute paths if we can see that the vagrant has mapped, mounted, or moved all the necessary files to the absolute locations specified in the guest's solo.rb configuration file?

@MartinSGill
Copy link

I updated my solo.rb to use proper windows paths:

if Chef::VERSION.to_f < 11.8
  role_path "c:/tmp/vagrant-chef-2/chef-solo-2/roles"
else
  role_path ["c:/tmp/vagrant-chef-2/chef-solo-2/roles"]
end

basically I add c: to the beginning and it now works correctly. Does this mean vagrant is generating the solo.rb incorrectly?

@lonniev
Copy link
Contributor Author

lonniev commented Dec 15, 2014

If solo.rb is parsed by all the same file I/O libraries that enable Ruby File to do the right magic on Windows, then I would assume that the chef-solo would translate the unixy paths in solo.rb to DOS ones correctly.

However, given your test, I now am uncertain.

Also, why would chef-solo be able to resolve the very similar cookbook path but not the role path?

That suggests that the solo.rb file is following the correct convention (leave all paths in Ruby-standard form) and it is code within chef-solo for the Windows platform that is not doing the transformation to DOS paths that the code also within chef-solo for the Windows platform does do for the cookbook path.

That then is a clue viable enough to lead one to open up the chef-solo code and simply compare the two bits of code.

@lonniev
Copy link
Contributor Author

lonniev commented Dec 15, 2014

It looks like in September code was added to all the file retrievers to do Dir.glob lookups for suitable files. Given this, it's likely that the lookups are now the same for each kind of file (cookbook, role, etc.).

What may be possible is that the run_list.expand method might be returning an error other than "file not found" and the code that raises the Chef::Exceptions::MissingRole exception only assumes that the reason expand would fail is because the file(s) could not be found.

I'm taking a look to see if expand might fail (on Windows, for role files) for other reasons than just a missing file. (For example, it could be that a too-long filename string or "-" characters in the filenames might cause a pattern matcher to return a false negative.)

@lonniev
Copy link
Contributor Author

lonniev commented Dec 15, 2014

@lamont-granquist is the current "owner" of RunListExpansion; he may be a wise mentor here.

Being chatty here:

    # Expand a run list from disk. Suitable for chef-solo
    class RunListExpansionFromDisk < RunListExpansion

      def fetch_role(name, included_by)
        Chef::Role.from_disk(name)
      rescue Chef::Exceptions::RoleNotFound
        role_not_found(name, included_by)
      end

    end

and

    # Load a role from disk - prefers to load the JSON, but will happily load
    # the raw rb files as well. Can search within directories in the role_path.
    def self.from_disk(name)
      paths = Array(Chef::Config[:role_path])
      paths.each do |path|
        roles_files = Dir.glob(File.join(Chef::Util::PathHelper.escape_glob(path), "**", "**"))

What this suggests is that @MartinSGill has suggested an immediate hackaround for us: simply add both the local relative unixy path and the mapped absolute dos path to the chef.roles_path array argument. Ideally, the hardcoded dos path SHOULD be unnecessary but the second loop of the above paths.each block would then find the sought file as it visited the hardcoded path with proper DOS path format.

@lonniev
Copy link
Contributor Author

lonniev commented Dec 15, 2014

https://raw.githubusercontent.com/opscode/chef/a93b2b67209db49ebbf9a52655894e3b7e150618/lib/chef/util/path_helper.rb

Where does the DOS drive letter get prefixed to the paths?

Also, the chef-solo file retrievers could validate that the configured paths are free of cruft characters and within the DOS length limit--using the check methods offered in this little path_helper class.

For now, I'll use the hackaround and give the owners of these files a chance to remark on what pilot errors I may be suffering or on how best to improve the identified source files.

@lonniev
Copy link
Contributor Author

lonniev commented Dec 15, 2014

Dang, vagrant is too smart: the array of paths in the Vagrantfile must be resolvable local paths and it drops from that array any paths which aren't parseable or existing on the Host OS. So, the hack addition isn't passed along in the solo.rb file.

@lonniev
Copy link
Contributor Author

lonniev commented Dec 15, 2014

Perhaps the Vagrantfile and the Chef code are both ok and the issue is related to the fact that the collection of chef directories under "tmp" is a collection of Windows symlinks to various \vboxsrv-located directories.

Can the Ruby File methods properly traverse those links or do they run into I/O errors as they try to open or stat the components of the paths?

@lamont-granquist
Copy link
Contributor

I don't think the changes to RunListExpansion should have affected this. It looks more like you've got issues in Chef::Config dealing with how the role_path is expanded via Chef::Util::PathHelper stuff, and I'm not the expert there. The "escape_glob()" function is necessary on windows when using Dir.glob and needs to not include the actual glob characters, so that function looks correct. It matches the same usage of that function that we have elsewhere in the provider code.

There also seems to be some confusion in this issue around the use of role_path vs. roles_path as the configuration option and passing an array vs. a string into the config. The correct argument for chef is role_path not roles_path. In Chef 12 you should also be able to simply set chef_repo_path to the parent directory and have everything resolved to subdirs of that. The intent is that both arrays and strings will work for all of them. Sometimes there's other external gems which parse Chef::Config that haven't been updated for that usage, although the latest Berkshelf/ridley has been updated for that. I know knife-spork also made assumptions about cookbook_path being either a string or array which got recently fixed as well.

@sethvargo
Copy link
Contributor

@lamont-granquist 66d8c85

@lonniev
Copy link
Contributor Author

lonniev commented Dec 15, 2014

@lamont-granquist I think you are right about the source of the issue being within either Chef::Util::PathHelper or back with roles/role_path attribute.

Note I was using chef.roles_path in my Vagrantfile. When I switch that to chef.role_path, vagrant then complains:

Lonnies-MacBook-Pro:virtualbox-tasktopsync lonniev$ vagrant provision vb-tt-sync
There are errors in the configuration of this machine. Please fix
the following errors and try again:

chef solo provisioner:
* The following settings shouldn't exist: role_path
* The following settings shouldn't exist: role_path

So, I am forced to use roles_path and it does set role_path in the solo.rb.

@lonniev
Copy link
Contributor Author

lonniev commented Dec 15, 2014

@sethvargo seems to be confirming this unfortunate use of the attribute names roles_path to role_path.

@lonniev
Copy link
Contributor Author

lonniev commented Dec 15, 2014

It is just a smoky day in the Chef kitchens. More burned cr*p:

Lonnies-MacBook-Pro:virtualbox-tasktopsync lonniev$ vagrant plugin update
Updating installed plugins...
Updated 'chef' to version '12.0.1'!
Updated 'vagrant-aws' to version '0.6.0'!
Updated 'vagrant-azure' to version '1.0.5'!
Updated 'vagrant-share' to version '1.1.4'!
Lonnies-MacBook-Pro:virtualbox-tasktopsync lonniev$ vagrant provision vb-tt-sync
==> vb-tt-sync: Installing Chef cookbooks with Librarian-Chef...
/Users/lonniev/.vagrant.d/gems/gems/vagrant-azure-1.0.5/lib/vagrant-azure/communication/powershell.rb:24:in `ready?': undefined method `check_winrm' for #<VagrantPlugins::ProviderVirtualBox::Driver::Meta:0x000001041f59c8> (NoMethodError)
    from /Users/lonniev/.vagrant.d/gems/gems/vagrant-omnibus-1.4.1/lib/vagrant-omnibus/action/install_chef.rb:40:in `call'

@sethvargo
Copy link
Contributor

@sneal ^. Halp.

@lamont-granquist
Copy link
Contributor

Welp vagrant-azure, i'm outta here... good luck!! lol...

Paging @Adamex and @randomcamel and @jdmundrawala who might have more useful knowledge this deep into chef+windows. I'm way over my head at this point.

All I can think of is that it might be better to try Berkshelf rather than Librarian and see if that works any better. TK will eventually support windows/azure (soon), but doesn't right this moment so that isn't an alternative. I'm in over my head tho.

@lonniev
Copy link
Contributor Author

lonniev commented Dec 15, 2014

Np, Lamont. I don't think it's a librarian chef thing because all librarian does is populate the cookbook directories on the host in an automated way. Once chef-solo is off and running, librarian isn't part of the process. It is due, I think, to something with Windows path names and perhaps symlinks.

—Lonnie VanZandt

303-900-3048
Sent from Dropbox's Mailbox on Mac

On Mon, Dec 15, 2014 at 1:28 PM, Lamont Granquist
notifications@github.com wrote:

Welp vagrant-azure, i'm outta here... good luck!! lol...
Paging @Adamex and @randomcamel and @jdmundrawala who might have more useful knowledge this deep into chef+windows. I'm way over my head at this point.

All I can think of is that it might be better to try Berkshelf rather than Librarian and see if that works any better. TK will eventually support windows/azure (soon), but doesn't right this moment so that isn't an alternative. I'm in over my head tho.

Reply to this email directly or view it on GitHub:
#4974 (comment)

@sneal
Copy link
Contributor

sneal commented Dec 16, 2014

@sethvargo You're right. I filed a new Chef issue chef/chef#2666 so at least they're aware of the behavior change.

@lonniev
Copy link
Contributor Author

lonniev commented Dec 16, 2014

@sethvargo, no problem, I understand. Yes, I know the drive letter is the hard part (I’m a former unix filesystem developer) because Unix has a nice unified filesystem architecture while Windows doesn’t. If I was writing #to_osFormat(), I might accept an optional hash of OS-specific junk. Perhaps therein I’d allow a drive tag to default to “c:\”. Making that assumption seems to be pervasive in ruby/vagrant/chef land. It might also be deduced from %USER% or %WINNT% or some other probably-safe assumption.

I’m mainly surprised that this is a new issue.

Anyway, thanks guys for attending to it: I’ll await progress and I won’t trivialize what you are doing.

@lamont-granquist
Copy link
Contributor

I suspect this is probably a Chef 12 regression where we tightened up what we accepted but weren't considering this use case, and I've marked the associated chef bug as such.

@lonniev
Copy link
Contributor Author

lonniev commented Dec 18, 2014

I am able to work around the role_path lack-of-a-drive-letter-prefix issue chef/chef#2666 with this workaround:

in the Vagrantfile, within the block for the chef provisioner, add:

chef.custom_config_path = "ChefConfig.WindowsMonkeyPatch"

Then, add the new file ChefConfig.WindowsMonkeyPatch and provide the following content:

role_path "c:/tmp/vagrant-chef-2/chef-solo-2/roles"

This will replace the (mis)computed value of the role_path setting with the known actual well-formed absolute Windows path to that same directory.

Note that you may name the file anything legitimate and that the absolute Windows path will depend on the order of other chef.*_path lines in your provisioner block. To determine the right absolute path, examine the runtime VM when the chef.role_path "roles" line is specified and then use the actual path discovered in the running Windows VM after the MissingRole exception is thrown.

@sethvargo
Copy link
Contributor

Ohai! I'm going to close this as we have determined this is a regression in Chef 12. If Chef determines this is "not a bug", we can revisit potential ways to fix this in Vagrant, but this is really an upstream bug.

@sidd-kulk
Copy link

I am getting the exact same issue on Mac OSX. Any solution?

@yeungs
Copy link

yeungs commented Apr 9, 2015

FYI. I am hitting this issue as well @lonniev . My setup is Vagrant 1.7.2 on Windows 2012r2 using Microsoft Hyper-V as provider to create Windows 2012r2 VM with Chef-solo 12.2.1 (apparently I was being silly to think latest is greatest) in that VM.

I was happy with Vagrant 1.7.1 with chef-solo 11.14.2 . The only reason I moved is because the "sql_server" cookbook reveal a bug in the older version of chef-solo. :( That was my day.

I am just glad someone knows about this.

@yeungs
Copy link

yeungs commented Apr 9, 2015

I just downgraded to Chef client version 11.18.6 and the role problem went away. 👍

Hopefully this helps others.

@NickMRamirez
Copy link

This doesn't look like a bug in Chef to me. Rather, it's a bug in the default value Vagrant assigns to the provisioning_path config property.

By default, this property is set to /tmp/chef-vagrant, even on Windows guests.

Shared Chef Options:

provisioning_path (string) - The path on the remote machine where Vagrant will store all necessary
files for provisioning such as cookbooks, configurations, etc. This path must be world writable. By
default this is /tmp/vagrant-chef-# where "#" is replaced by a unique counter.

/tmp/vagrant-chef sounds pretty Linux-y. I don't get the error if I set that config property to something more Windows-y:

config.vm.provision 'chef_zero' do |chef|
    chef.provisioning_path = 'C:\vagrant-chef'

    chef.roles_path = 'roles'
    chef.add_role 'windowsnode'
end

Maybe something along the lines of updating the default value of provisioning_path in vagrant/plugins/provisioners/chef/config/base_runner.rb if the guest is Windows? I can open a new issue for this.

@lonniev
Copy link
Contributor Author

lonniev commented Jun 24, 2015

The path is fine. This works in chef 12.4.0.rc.1

Generally, the issues all arise with dealing with Windows pathname drive letters and backslashes that get interpreted by Ruby regex as escaping codes.

(I need to turn off a Chef Config monkey patch to assure that 12.4.0.rc.1 gets it right but I haven’t had the role issue myself for a week or so with 12.4.0.rc.1 — but the patch may still be doing its workaround. Your conversation here is a good kick to me to go see if I still need it.)

-- 
Lonnie VanZandt
303-900-3048

On 24 June 2015 at 13:42:25, NickMRamirez (notifications@github.com) wrote:

This doesn't look like a bug in Chef to me. Rather, it's a bug in the default value Vagrant assigns to the provisioning_path config property.

By default, this property is set to /tmp/chef-vagrant, even on Windows guests.

Shared Chef Options:

provisioning_path (string) - The path on the remote machine where Vagrant will store all necessary
files for provisioning such as cookbooks, configurations, etc. This path must be world writable. By
default this is /tmp/vagrant-chef-# where "#" is replaced by a unique counter.

/tmp/vagrant-chef sounds pretty Linux-y. I don't get the error if I set that config property to something more Windows-y:

config.vm.provision 'chef_zero' do |chef|
chef.provisioning_path = 'C:\vagrant-chef'

chef.roles_path = 'roles'
chef.add_role 'windowsnode'

end
Maybe something along the lines of updating the default value of provisioning_path in vagrant/plugins/provisioners/chef/config/base_runner.rb if the guest is Windows? I can open a new issue for this.


Reply to this email directly or view it on GitHub.

@NickMRamirez
Copy link

Hmm...I wonder what they're doing in the release candidate. Because even escaped,

\tmp\vagrant-chef

doesn't get interpreted right by Windows. Really, /tmp isn't a normal Windows directory. Putting a drive letter at the front makes the problem go away, via File.absolute_path or File.expand_path, but the feedback I got when I submitted a pull request in Chef that does that was that there might be unintended consequences if the user changes the running process' current working directory.

chef/chef#3569

@lonniev
Copy link
Contributor Author

lonniev commented Jun 24, 2015

The vagrant base box I built for SoftLayer does create a c:\tmp directory. I need to check my state of affairs with what I have working with 12.4.0.rc.1. I may just be lucky.

-- 
Lonnie VanZandt
303-900-3048

On 24 June 2015 at 14:01:28, NickMRamirez (notifications@github.com) wrote:

Hmm...I wonder what they're doing in the release candidate. Because even escaped,

\tmp\vagrant-chef

doesn't get interpreted right by Windows. Really, /tmp isn't a normal Windows directory. Putting a drive letter at the front makes the problem go away, via File.absolute_path or File.expand_path, but the feedback I got when I submitted a pull request in Chef that does that was that there might be unintended consequences if the user changes the running process' current working directory.


Reply to this email directly or view it on GitHub.

@NickMRamirez
Copy link

I opened another issue for the Vagrant default paths, just to see what the consensus is:

#5862

@lonniev
Copy link
Contributor Author

lonniev commented Jun 25, 2015

I was just lucky. If I remove the line:

        chef.custom_config_path = "ChefConfig.WindowsMonkeyPatch"

where the following is set on the guest:

role_path "c:/vagrant/roles"

Then chef role-specified provisioning is busted because the roles path can't be resolved.

I apologize for being optimistic; I thought I had removed this bandaid.

@lonniev
Copy link
Contributor Author

lonniev commented Jun 25, 2015

Darn fragile code:

rsync: mkdir "/cygdrive/c/Users/vagrant/C:vagrant-chef/5e5b4497aa57800c3fe732202ad0e04e/cookbooks" failed: No such...

That's what I get when I use the provisioning_path fix you suggest. It looks like for now I am stuck with my little monkey patch. (We can see that the rsync handling code doesn't know how to cygwin-ize a ruby path for Windows that already has drive letters within it.)

@NickMRamirez
Copy link

@lonniev For cygwin, what's the setup? I'd like to try some things out with it. (or with rsync)

@lonniev
Copy link
Contributor Author

lonniev commented Jun 25, 2015

It isn’t the case that a guest VM happens to have cygwin installed onto but rather than the rsync executable most usable for Windows was built with Cygwin’s libraries for dealing with Windows pathnames and other OS-specific details. Therefore, the vagrant and chef modules that call rsync are responsible for transforming Ruby paths to Windows paths to Cygwin paths.

-- 
Lonnie VanZandt
303-900-3048

On 25 June 2015 at 05:33:51, NickMRamirez (notifications@github.com) wrote:

@lonniev For cygwin, what's the setup? I'd like to try some things out with it.


Reply to this email directly or view it on GitHub.

@NickMRamirez
Copy link

I tried using MinGW to get rsync installed, but couldn't get it working at all with Vagrant. (unrelated to Chef provisioning). I'd never tried before and am just going off of the Vagrant rsync docs. Maybe you can share your Vagrantfile and setup?

My Vagrantfile otherwise, solves the Windows guest problems I was having before.

Vagrant.configure(2) do |config|

  config.vm.box = "kensykora/windows_2012_r2_standard"

  # Could not get rsync working...
  # config.vm.synced_folder '.', '/my_share', type: 'rsync'

  config.vm.provision 'chef_zero' do |chef|
    chef.provisioning_path = 'C:/vagrant-chef' # note forward slashes OK
    chef.file_backup_path = 'C:/chef/backup'
    chef.file_cache_path = 'C:/chef/cache'

    chef.roles_path = 'roles'
    chef.add_role 'windowsnode'
  end
end

If we can get the rsync/cygwin stuff figured out, would make a better case for changing the default paths for Windows.

@lonniev
Copy link
Contributor Author

lonniev commented Jun 25, 2015

The following comes from a Packer script for making vagrantized Windows boxes. One also has to setup OpenSSH on the image beforehand.

rem install rsync
if not exist "C:\Windows\Temp\7z920-x64.msi" (
    powershell -Command "(New-Object System.Net.WebClient).DownloadFile('http://downloads.sourceforge.net/sevenzip/7z920-x64.msi', 'C:\Windows\Temp\7z920-x64.msi')" <NUL
)
msiexec /qb /i C:\Windows\Temp\7z920-x64.msi

pushd C:\Windows\Temp
powershell -Command "(New-Object System.Net.WebClient).DownloadFile('http://mirrors.kernel.org/sourceware/cygwin/x86_64/release/rsync/rsync-3.1.0-1.tar.xz', 'C:\Windows\Temp\rsync-3.1.0-1.tar.xz')" <NUL
cmd /c ""C:\Program Files\7-Zip\7z.exe" x rsync-3.1.0-1.tar.xz"
cmd /c ""C:\Program Files\7-Zip\7z.exe" x rsync-3.1.0-1.tar"
copy /Y usr\bin\rsync.exe "C:\Program Files\OpenSSH\bin\rsync.exe"
rmdir /s /q usr
del rsync-3.1.0-1.tar
popd

msiexec /qb /x C:\Windows\Temp\7z920-x64.msi

rem make symlink for c:/vagrant share
mklink /D "C:\Program Files\OpenSSH\vagrant" "C:\vagrant"

@NickMRamirez
Copy link

I got rsync working with a Windows host and Windows guest. Steps I took:

  1. Installed cygwin with its OpenSSH and rsync packages onto the host machine.
  2. Used the following Vagrantfile that uses kensykora/windows_2012_r2_standard box (comes with OpenSSH already installed). The Vagrantfile uses rsync fix, documented at Rsync on Windows using cwRsync fails with No such file or directory (2) #3230 (comment)
# rsync fix:
ENV["VAGRANT_DETECTED_OS"] = ENV["VAGRANT_DETECTED_OS"].to_s + " cygwin"

Vagrant.configure(2) do |config|

  config.vm.box = "kensykora/windows_2012_r2_standard"
  config.vm.network 'private_network', ip: '192.168.50.4'

  # Note: guest path uses /cygdrive, even though cygwin is only installed on the host
  config.vm.synced_folder './my_share', '/cygdrive/c/my_share', type: 'rsync'

  config.vm.provision 'chef_zero' do |chef|
    chef.provisioning_path = 'C:/vagrant-chef'
    chef.file_backup_path = 'C:/chef/backup'
    chef.file_cache_path = 'C:/chef/cache'

    chef.roles_path = 'roles'
    chef.add_role 'windowsnode'
  end
end
  1. After vagrant up, vagrant rdp'ed into the machine so I could install rsync.
  2. Copied your commands for installing rsync to the guest into a CMD file on the guest and ran it. This installed rsync (The box already had OpenSSH installed). This could be skipped if the guest already had rsync.
  3. Ran vagrant reload

That did it to sync up the my_share folders.

@lonniev
Copy link
Contributor Author

lonniev commented Aug 8, 2015

Status update: the combination of MacOS, Vagrant, Chef-Solo, Windows still has, in Vagrant 1.7.4 and Chef 12.4.1, issues with rsync and Windows paths.

I should create a new issue. If I recall, it is the Windows Capabilities Rsync ruby code in core Vagrant that is responsible for getting this resolved.

There was an error when attempting to rsync a synced folder.
Please inspect the error message below for more info.

Host path: /Users/lonniev/Vagrants/softlayer-windows-obeoteam/cookbooks/
Guest path: C:/vagrant-chef/f3c765f92a8356d41d4338281bc64a00/cookbooks
Command: rsync --verbose --archive --delete -z --copy-links --no-owner --no-group -e ssh -p 22 -o StrictHostKeyChecking=no -o IdentitiesOnly=true -o UserKnownHostsFile=/dev/null -i '/Users/lonniev/.vagrant.d/insecure_private_key' --exclude .vagrant/ /Users/lonniev/Vagrants/softlayer-windows-obeoteam/cookbooks/ vagrant@255.255.255.255:C:/vagrant-chef/f3c765f92a8356d41d4338281bc64a00/cookbooks
Error: Warning: Permanently added '255.255.255.255' (RSA) to the list of known hosts.
cygwin warning:
  MS-DOS style path detected: C:/vagrant-chef/f3c765f92a8356d41d4338281bc64a00/cookbooks
  Preferred POSIX equivalent is: /cygdrive/c/vagrant-chef/f3c765f92a8356d41d4338281bc64a00/cookbooks
  CYGWIN environment variable option "nodosfilewarning" turns off this warning.
  Consult the user's guide for more details about POSIX paths:
    http://cygwin.com/cygwin-ug-net/using.html#using-pathnames
rsync: change_dir#1 "/cygdrive/c/Users/vagrant//C:/vagrant-chef/f3c765f92a8356d41d4338281bc64a00/cookbooks" failed: No such file or directory (2)
rsync error: errors selecting input/output files, dirs (code 3) at main.c(631) [Receiver=3.1.0]

# for free to subscribe to this conversation on GitHub. Already have an account? #.
Projects
None yet
Development

No branches or pull requests

8 participants