Default logger, or a way to share logger configuration between modules

[rprieto]

Context: we split our app into a main process, and several smaller "npm" modules, some of which use bunyan.

For now, these modules output to the default stdout stream, in which case all the logs get aggregated properly. But the main app might configure bunyan to output to a file instead. How would we share the "main" logger configuration, so that all modules using Bunyan start outputting to the same place?

For example:

main-app.js

var bunyan = require('bunyan');

var log = bunyan.createLogger({
  name: 'myapp',
  streams: [{ level: 'debug', path: 'app.log' }]
});

log.info('app starting');
require('module-b').doStuff();

module-b

var bunyan = require('bunyan');
/* if we don't do anything special, this will just go to the console */

exports.doStuff = function() {
   log.info('module B doing stuff');
};

A basic way would be to export an initLogger function in module-b, and pass down the configuration from our app, but I was hoping for a cleaner way - maybe some global logger config that can be stored and shared?

[rlidwka]

Create a module named "logger.js" and require it from anywhere you want.

[rprieto]

Thanks! Just to clarify, module-b is not a module in the "file" sense, but in the npm install module-b sense.

If creating a logger module that all others should depend on, I'm happy to do that.

I thought Bunyan could have an option to define defaults for all loggers in memory. This might be even more relevant for open-source modules. If we depend on open-source-module-x which uses Bunyan, I would be nice if we can configure it to comply to our app's configuration, so its output is written to the same log file for example.

[aronrodrigues]

A clean solution would be a function to change the default stream.

var bunyan = require('bunyan');
bunyan.changeDefaultStream([
{
level: 'info',
stream: process.stdout
},
{
level: 'error',
path: '/var/log/myapp-error.log'
}
]);

[aronrodrigues]

A way to bypass it is to define new modules on the main file and use it instead.

var bunyan = require("bunyan");
bunyan.defaultStreams = [
  {
    level: 'info',
    stream: process.stdout

},
  {
    level: 'error',
    path: '/var/log/myapp-error.log'
}
];
bunyan.getLogger(name) {
  if (defaultStreams) {
    return bunyan.createLogger(name: name, streams: bunyan.defaultStreams);
  } else {
    return bunyan.createLogger(name: name);
  }

on the module file use getLogger instead createLogger.

var bunyan = require("bunyan");
bunyan.getLogger('moduleA');

[trentm]

I guess this is similar to the idea in log4j (likewise clones like log4js, Python's logging module, etc.) where logger objects are global to the process such that, e.g., log = logging.getLogger("foo") used in two separate modules gets the same logger. My experience in that world is that typically different modules use different logger names... but they inherit from the global logger on which output streams are typically configured. (Oh, I think this is what @aronrodrigues described.)

There was some bunyan wrapper module on npm that I saw a long while back that allowed for globally shared loggers. I can't recall the name right now. Found it: https://www.npmjs.org/package/bole

Personally, I my usage, I get all modules to take a log option which is a Bunyan logger. So something like:

// app.js
var log = bunyan.createLogger(...);

var SomeModuleThing = require('some-module');
var thing = new SomeModuleThing({log: log, ...});
// use thing

and each module might use log.child to have log records show that it is from that module:

// some-module.js
module.exports = function SomeModuleThing(opts) {
   this.log = opts.log.child({someModule: true});
};

IOW, I write modules to say: "Oh, you want me to log somewhere, then pass me the logger on which to do it." I'll grant that this does mean an impact on APIs in that you have to pass this 'log' param around. I've not thought about including a sense of globally shared loggers (by name) in Bunyan core.

[aronrodrigues]

I think that just an option to overwrite the default stream is enough.
If you want, I can fork the project and build it. It is less than 10
lines.
On Sep 28, 2014 12:45 AM, "Trent Mick" notifications@github.com wrote:

I guess this is similar to the idea in log4j (likewise clones like log4js,
Python's logging module, etc.) where logger objects are global to the
process such that, e.g., log = logging.getLogger("foo") used in two
separate modules gets the same logger. My experience in that world is
that typically different modules use different logger names... but they
inherit from the global logger on which output streams are typically
configured. (Oh, I think this is what @aronrodrigues
https://github.com/aronrodrigues described.)

There was some bunyan wrapper module on npm that I saw a long while back
that allowed for globally shared loggers. I can't recall the name right
now. Found it: https://www.npmjs.org/package/bole

Personally, I my usage, I get all modules to take a log option which is
a Bunyan logger. So something like:

// app.jsvar log = bunyan.createLogger(...);
var SomeModuleThing = require('some-module');var thing = new SomeModuleThing({log: log, ...});// use thing

and each module might use log.child to have log records show that it is
from that module:

// some-module.jsmodule.exports = function SomeModuleThing(opts) {
this.log = opts.log.child({someModule: true});};

IOW, I write modules to say: "Oh, you want me to log somewhere, then pass
me the logger on which to do it." I'll grant that this does mean an
impact on APIs in that you have to pass this 'log' param around. I've not
thought about including a sense of globally shared loggers (by name) in
Bunyan core.

—
Reply to this email directly or view it on GitHub
#116 (comment).

[trentm]

Each require('Bunyan') will be independent so that doesn't help unless we are proposing this reach into process global state. If that then I'd want to think about an API for each module to use that makes it clear it is process global.

On Sep 27, 2014, at 9:03 PM, aronrodrigues notifications@github.com wrote:

I think that just an option to overwrite the default stream is enough.
If you want, I can fork the project and build it. It is less than 10
lines.
On Sep 28, 2014 12:45 AM, "Trent Mick" notifications@github.com wrote:

I guess this is similar to the idea in log4j (likewise clones like log4js,
Python's logging module, etc.) where logger objects are global to the
process such that, e.g., log = logging.getLogger("foo") used in two
separate modules gets the same logger. My experience in that world is
that typically different modules use different logger names... but they
inherit from the global logger on which output streams are typically
configured. (Oh, I think this is what @aronrodrigues
https://github.com/aronrodrigues described.)

There was some bunyan wrapper module on npm that I saw a long while back
that allowed for globally shared loggers. I can't recall the name right
now. Found it: https://www.npmjs.org/package/bole

Personally, I my usage, I get all modules to take a log option which is
a Bunyan logger. So something like:

// app.jsvar log = bunyan.createLogger(...);
var SomeModuleThing = require('some-module');var thing = new SomeModuleThing({log: log, ...});// use thing

and each module might use log.child to have log records show that it is
from that module:

// some-module.jsmodule.exports = function SomeModuleThing(opts) {
this.log = opts.log.child({someModule: true});};

IOW, I write modules to say: "Oh, you want me to log somewhere, then pass
me the logger on which to do it." I'll grant that this does mean an
impact on APIs in that you have to pass this 'log' param around. I've not
thought about including a sense of globally shared loggers (by name) in
Bunyan core.

—
Reply to this email directly or view it on GitHub
#116 (comment).

—
Reply to this email directly or view it on GitHub.

[n4nagappan]

// logger.js

var bunyan = require('bunyan');
var obj = {};

if( !obj.log ){
  obj.log = bunyan.createLogger({name : "categories"});
}
module.exports = obj;

Use this in all modules as
var log = require('/path/to/logger.js').log;

[jamelt]

@n4nagappan the good ol' singleton pattern

[dharmax]

The only thing that worked for me, for some reason, is to write the loggers into an object under the global variable;

global.loggers = { 
       logger1: bunyan.createLogger(...)
}

i use typescript, so i use 'import', not require, and it gave me lots of pain to make it work...reason is unknown, really.

[servel333]

I solved this problem by writing a custom addChildLogger that will add a stream to the child that forwards all logs to the parent logger. This version will print all messages from the child to the parent.

const addChildLogger = function(parent, child){
  child.addStream({
    type: 'raw',
    stream: { write: function(rec){ parent._emit(rec); } },
    level: 'trace',
  });
};

addChildLogger(parentLogger, moduleLogger);

Here is a version that supports log level, but it's untested so make sure there are no typos or whatnot.

const addChildLogger = function(parent, child, level){
  child.addStream({
    type: 'raw',
    stream: {
      write: function(rec){
        if (this._level <= rec.level) {
          parent._emit(rec);
        }
      }
    },
    level: level || 'trace',
  });
};

addChildLogger(parentLogger, moduleLogger, "trace");

[reachtrevor]

@rprieto, I hope I am understanding the question correctly but here's my logger configuration. This setup allows me to import the global settings anywhere AND create child loggers off of it to identify different parts of the application.

Note: I am using typescript so ignore if you wish.

config.yml

logger:
    basePath:            ./log/
    level:                    info
    name:                   main-api
    serviceLog:          service.log

logUtils.ts

import * as path from 'path';
import * as bunyan from 'bunyan';

import { Stream } from 'bunyan';
import conf from '../config';

let mainLoggerStreams: Stream[] = [{
  stream: process.stdout
}];

if (conf.env === 'production') {
  mainLoggerStreams = [{
    path: path.resolve(conf.logger.basePath, 'service.log')
  }];
}

// main logger
const logger = bunyan.createLogger({
  name: conf.logger.name,
  level: conf.logger.level,
  serializers: bunyan.stdSerializers,
  streams: mainLoggerStreams
});

export default logger;

Then in a controller somewhere...
controller

import logger from '../utils/logUtils';
const log = logger.child({ feature: 'events' });

// somewhere in the controller
log.info('something happened');
log.error('oh noes!');

All of the logs in the controller will have an extra feature key on them which allow me to use the bunyan cli to filter the results by feature.

Hope this helps!