Implement a C.I build

[mattwarren]

Maybe using AppVeyor, i.e. like this https://ci.appveyor.com/project/mattwarren/minibench-warren. This way if someone (probably me) breaks the build or some tests, we'll know about it sooner rather than later

See this discussion for some background and extra info.

[mattwarren]

I've started working on this, however I've hit an issue with dotnet restore @adamsitnik are you able to offer any ideas?

Basically I'm running:

choco install dotnetcore -version 1.0.0.001598
set PATH=C:\Program Files\dotnet\bin;%PATH%
dotnet restore --source https://api.nuget.org/v3/index.json BenchmarkDotNet
dotnet build BenchmarkDotNet

However dotnet restore fails with:

...
error: Unable to resolve NETStandard.Library (>= 1.0.0-rc2-23811) for DNXCore,Version=v5.0.
info : Committing restore...
log  : Restore failed in 2500ms.

Errors in C:\projects\benchmarkdotnet-8ww0x\BenchmarkDotNet\project.json
    Unable to resolve NETStandard.Library (>= 1.0.0-rc2-23811) for DNXCore,Version=v5.0.

NuGet Config files used:
    C:\Users\appveyor\AppData\Roaming\NuGet\NuGet.Config

Feeds used:
    https://api.nuget.org/v3/index.json

Installed:
    34 package(s) to C:\projects\benchmarkdotnet-8ww0x\BenchmarkDotNet\project.json
Command exited with code 1

Any ideas?

The full log is available if that helps?

[AndreyAkinshin]

@mattwarren, i think you should use https://dotnet.myget.org/F/dotnet-core/api/v3/index.json as a NuGet feed for dnxcore.

[adamsitnik]

@mattwarren You need another nuget feed for .NET Core packages. In .nuge\NuGet.config you can find the whole list.

You can run:
dotnet restore --source https://api.nuget.org/v3/index.json --fallbacksource https://dotnet.myget.org/F/dotnet-core/api/v3/index.json BenchmarkDotNet

[adamsitnik]

@mattwarren Another problem you probably will face is dnx/dotnet cli differences. In general VS is still using dnx to build all project.json-based solutions (switch to dotnet cli is planned for RC2). And I am not sure if xUnit runner supports dotnet cli at all.

How I do "core CI things" at work:

1. Step: install dnvm
   @powershell -NoProfile -ExecutionPolicy unrestricted -Command "&{$Branch='dev';iex ((new-object net.webclient).DownloadString('https://raw.githubusercontent.com/aspnet/Home/dev/dnvminstall.ps1'))}"
2. Dnx install:
   dnvm upgrade -r clr
3. Packages restore:
   dnu restore
4. Running tests (it builds the project as well)
   cd to folder with tests dnx --configuration RELEASE test

@AndreyAkinshin I think that we should also add some CI for Mono. It would be great if your verification before nuget package release could be 100% automated.

[mattwarren]

Thanks to both of you for the tip about --fallbacksource https://dotnet.myget.org/F/dotnet-core/api/v3/index.json that worked (takes a long time to restore though 239,423ms! I hope AppVeyor does some caching and isn't downloading all that for every build!!

However I now have a problem related to MsBuild, seems like MsBuild 12.0 isn't installed:

Compiling BenchmarkDotNet for .NETFramework,Version=v4.5 
C:\Program Files\dotnet\bin\dotnet.exe compile-csc @C:\projects\benchmarkdotnet-8ww0x\BenchmarkDotNet\obj\Debug\net45\dotnet-compile.rsp returned Exit Code 1 
C:\projects\benchmarkdotnet-8ww0x\BenchmarkDotNet\project.json(35,28): warning NU1007: Dependency specified was Microsoft.Build >= 12.0.0 but ended up with Microsoft.Build 4.0.0.0. 
C:\projects\benchmarkdotnet-8ww0x\BenchmarkDotNet\project.json(36,38): warning NU1007: Dependency specified was Microsoft.Build.Framework >= 12.0.0 but ended up with Microsoft.Build.Framework 4.0.0.0. 
C:\projects\benchmarkdotnet-8ww0x\BenchmarkDotNet\project.json(37,44): error NU1001: The dependency Microsoft.Build.Utilities.v12.0 >= 12.0.0 could not be resolved. 
C:\projects\benchmarkdotnet-8ww0x\BenchmarkDotNet\BenchmarkDotNet\Loggers\MSBuildConsoleLogger.cs(3,23): error CS0234: The type or namespace name 'Utilities' does not exist in the namespace 'Microsoft.Build' (are you missing an assembly reference?) 
C:\projects\benchmarkdotnet-8ww0x\BenchmarkDotNet\BenchmarkDotNet\Loggers\MSBuildConsoleLogger.cs(7,43): error CS0246: The type or namespace name 'Logger' could not be found (are you missing a using directive or an assembly reference?) 
C:\projects\benchmarkdotnet-8ww0x\BenchmarkDotNet\BenchmarkDotNet\Loggers\MSBuildConsoleLogger.cs(16,30): error CS0115: 'MsBuildConsoleLogger.Initialize(IEventSource)': no suitable method found to override 

Compilation failed. 
    2 Warning(s) 
    4 Error(s) 

Time elapsed 00:00:01.7242660 

Command exited with code 1

[mattwarren]

@mattwarren Another problem you probably will face is dnx/dotnet cli differences. In general VS is still using dnx to build all project.json-based solutions (switch to dotnet cli is planned for RC2). And I am not sure if xUnit runner supports dotnet cli at all.

I was planning to just use dotnet and see if that would work out for the time being. I didn't want to worry about dnx with it being retired sometime soon (maybe).

But if it's the only way I'll use those instructions, thanks for them

[adamsitnik]

However I now have a problem related to MsBuild, seems like MsBuild 12.0 isn't installed:

Is it possible to have AppVeyor instance with VS 2013 and VS 2015? Please let me know if you have any other problems.

[mattwarren]

I've looked at this a bit more:

• dotnet build BenchmarkDotNet --framework dnx451 works
• dotnet build BenchmarkDotNet --framework dnxCore works
• dotnet build BenchmarkDotNet --framework net40 works
• dotnet build BenchmarkDotNet --framework net451 FAILS

AppVeyor uses a VS 2015 image by default, but I also tried VS 2013 and got the same issue.

[adamsitnik]

Could you try net45 instead 451? And also net46?

[mattwarren]

net45 gives the same error as net451, see the log

net46 gives lots of errors, can't find System.String, System.Boolean, etc. So I've done something really wrong there!!

[adamsitnik]

I have simplified the dependencies by removing dependency to NETStandard.Library which is just huge a set of packages. Now the restore should be much more faster.

But I don't know what to do with the C.I. Perhaps we could try to use Team City? It is possible to get the license for free for OSS projects. If you don't mind I can register BDN and set up few build configurations: classic, dnx and core. What do you guys think? @mattwarren @AndreyAkinshin

[mattwarren]

@adamsitnik sorry I missed this comment.

But I don't know what to do with the C.I. Perhaps we could try to use Team City? It is possible to get the license for free for OSS projects. If you don't mind I can register BDN and set up few build configurations: classic, dnx and core. What do you guys think?

Fine by me, as long as one way or the other we get a C.I build, I don't mind which technology we use. I was only playing around with AppVeyor because I'd used it before.

[mattwarren]

There's a nice working AppVeyor here that we can use:

https://ci.appveyor.com/project/StackExchange/dapper-dot-net

[adamsitnik]

OK, now with #187 we have no more dnx dependencies, @mattwarren you should be able to restore, build and run tests with dotnet cli. You can take a look at the scripts that I wrote