Adding secrets as part of ConfigurationBuilder

[christcottrell]

Expected Behavior

I add configuration from from secrets through ConfigurationBuilder.

var environment = Environment.GetEnvironmentVariable("ASPNETCORE_ENVIRONMENT");
var configuration = new ConfigurationBuilder()
.SetBasePath(Directory.GetCurrentDirectory())
.AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
.AddJsonFile($"appsettings.{environment}.json", optional: true, reloadOnChange: true)
.AddDaprSecretStore("secret-store", daprClient)
.Build();

The issue is that these seems to happen before sidekick "kicks" in. So it is unable to find the secrets configuration. Because you add sidekick as part of ConfigureServices. I was able to work around this to setup the configuration as part of the

So what this should do is load all the secrets into the configuration so if you have let's say json configuration like this:
"SimpleConfigTest": {
"ConfigValue": "Not from secrets"
}
And in the secrets you have:
"SimpleConfigTest": {
"SecretValue": "From secrets"
}
And we could have like a SimpleConfigText object that is added to DI with the IOptions pattern.

services.Configure(configuration.GetSection("SimpleConfigTest"));

The biggest point is that this works with Dapr and the objects are initialized and works without sidekick.

To reproduce, create a secret store. I have a local secret store right now for development. Setup configuration to pull from the secret store.

Below is the stack trace seen.. Thanks!

at System.Net.Http.HttpClient.HandleFailure(Exception e, Boolean telemetryStarted, HttpResponseMessage response, CancellationTokenSource cts, CancellationToken cancellationToken, CancellationTokenSource pendingRequestsCts)
at System.Net.Http.HttpClient.<g__Core|83_0>d.MoveNext()
at Dapr.Client.DaprClientGrpc.d__60.MoveNext()
at Dapr.Client.DaprClientGrpc.d__61.MoveNext()
at Dapr.Extensions.Configuration.DaprSecretStore.DaprSecretStoreConfigurationProvider.d__16.MoveNext() in //src/Dapr.Extensions.Configuration/DaprSecretStoreConfigurationProvider.cs:line 193
at Dapr.Extensions.Configuration.DaprSecretStore.DaprSecretStoreConfigurationProvider.Load() in //src/Dapr.Extensions.Configuration/DaprSecretStoreConfigurationProvider.cs:line 184
at Microsoft.Extensions.Configuration.ConfigurationRoot..ctor(IList`1 providers) in //src/libraries/Microsoft.Extensions.Configuration/src/ConfigurationRoot.cs:line 36
at Microsoft.Extensions.Configuration.ConfigurationBuilder.Build() in //src/libraries/Microsoft.Extensions.Configuration/src/ConfigurationBuilder.cs:line 54
at Cmp.Api.Program.CreateWebHostBuilder(String[] args) in C:\projects\nova.api\Nova.Api\Program.cs:line 25
at Cmp.Api.Program.Main(String[] args) in C:\projects\nova.api\Nova.Api\Program.cs:line 14

[JonathanLydall]

We've also encountered this problem when trying to use Dapr Secrets with Sidekick.

The fundamental problem is that some services may need access to the secrets during startup, hence the docs advising that it be configured in the Program.cs file so that they're available, however, this happens before Sidekick has started the Dapr sidecar so it seems one can't generally use Sidekick and secrets at the same time, or if they do, then they need to be careful in their timing of when they try use secrets from IConfiguration.

Without having yet dug into this issue further, it seems that Sidekick needs to be able to start the Sidecar process inside the Program.cs file before secrets configuration and thus anything else needs to use the configuration (and I'm not sure how this interplays with Sidekick wanting to use configuration itself).

I'm also wondering on the health of this project considering that there has been no apparent activity on this GitHub repository in months.

[IliasP91]

I have the same problem as well, cant see any way to start it immediately at the beginning of the Program.cs

Might fork this and sort out the work dir issue as well as I would like to load secrets from local file secretstores for local dev and this doesnt allow it

[IliasP91]

Okay I have a little update. I managed to write a working solution using the existing released version of sidekick.
I used the DaprSidekickBuilder to start a Sidecar process earlier in the Program.cs, and then block the .net app until the sidecar components are initialized.
The sidekick hosted service later on will reuse the same process I started manually and everything will work fine reading the secrets during startup and even adding them in the configuration manager using the relevant extensions.

var builder = WebApplication.CreateBuilder(args);
var environment = builder.Environment;
var configuration = builder.Configuration;

// Manage local dapr sidecar
if (environment.IsDevelopment())
{
    var sidecarOptions = new DaprSidecarOptions
    {
        AppId = "my-service",
        AppPort = 5000,
        // Replace the following with ResourcesDirectory = Path.Combine(Environment.ContentRootPath, "Dapr/Components"), on the next release
        ComponentsDirectory = "./Dapr/Components",
        CustomArguments = $"--resources-path {Path.Combine(environment.ContentRootPath, "Dapr/Components")}",

        // Currently there's no option to override the working directory when the managed daprd process starts
        // so we copy it to the project directory to let the local components use relative paths
        ProcessFile = Path.Combine(environment.ContentRootPath, "daprd.exe"),
        CopyProcessFile = true
    };

    var sidekick = new DaprSidekickBuilder().Build();

    // Start the Dapr Sidecar imediately to get access to the components in startup
    sidekick.Sidecar.Start(
        () => new DaprOptions
        {
            LogLevel = "info",
            Sidecar = sidecarOptions
        },
        DaprCancellationToken.None);

    // Add Dapr Sidekick hosted service, it will attach and reuse the already started process above
    builder.Services.AddDaprSidekick(
        configuration,
        c =>
        {
            c.Sidecar = sidecarOptions;
        });
}

// Wait for the Dapr sidecar to report healthy before attempting to use any Dapr components.
using var tokenSource = new CancellationTokenSource(TimeSpan.FromMinutes(1));
await new DaprClientBuilder().Build().WaitForSidecarAsync(tokenSource.Token);

// Add custom configuration files.
configuration
    .SetBasePath(environment.ContentRootPath)
    .AddJsonFile("appsettings.json", false, true)
    .AddJsonFile($"appsettings.{environment.EnvironmentName}.json", true, true)
    .AddDaprSecretStore(SecretStores.KeyVault, new DaprClientBuilder().Build())
    .AddEnvironmentVariables();

// Test get secrets from store
var client = new DaprClientBuilder().Build();
var secrets = await client.GetSecretAsync(SecretStores.KeyVault, KeyVaultSecretKeys.MongoDbConnectionString);
Console.WriteLine($"----- Secret: {string.Join(",", secrets.Select(d => d.Value))}");

// Test get secrets from configs
Console.WriteLine($"----- Secret from configs: {configuration.GetSection(KeyVaultSecretKeys.MongoDbConnectionString).Value}");


...etc

[IliasP91]

relevant PR #59

[badgeratu]

Just a quick note (and apology) for tardiness of responses from the maintainers on this issue. While we do consider this project maintained and active, Sidekick has been working fine for us internally for a while so we haven't needed to make any changes and there have been workarounds for most reported issues. We'll continue to accept and review contributions/PR's and prepare point releases including those changes as needed. Additionally we are about to start evaluating newer versions of Dapr and its features which will no doubt lead to some enhancements (such as proper support for bootstrapping secrets).