Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

HTTP API: Logging #574

Closed
4 of 9 tasks
hsanjuan opened this issue Oct 11, 2018 · 10 comments · Fixed by #894
Closed
4 of 9 tasks

HTTP API: Logging #574

hsanjuan opened this issue Oct 11, 2018 · 10 comments · Fixed by #894
Labels
exp/wizard Extensive knowledge (implications, ramifications) required help wanted Seeking public contribution on this issue kind/enhancement A net-new feature or improvement to an existing feature P2 Medium: Good to have, but can wait until someone steps up
Milestone

Comments

@hsanjuan
Copy link
Collaborator

hsanjuan commented Oct 11, 2018

Basic information

  • Version information (mark as appropiate):
    • Master
    • Release candidate for next version
    • Latest stable version
    • An older version I should not be using
  • Type (mark as appropiate):
    • Bug
    • Feature request
    • Enhancement

Description

Our HTTP API currently does not do any logging that includes request, return codes, duration etc... (webserver-like). We should enable this just like any other webserver. We should also have a way to log, in the case of authorized requests (with basic auth), which requests were performed by which user.

We likely want these logs to be specific to the API component and have them written out separately from the usual output, and stored somewhere (/var/log/ipfs-cluster?).

To discuss:

  • Where to put these logs (if anywhere)
  • If this should be enabled by default
  • If this compels us to start using https://echo.labstack.com/ (and rewriting the API with it?)

cc. @lanzafame

@hsanjuan hsanjuan added kind/enhancement A net-new feature or improvement to an existing feature help wanted Seeking public contribution on this issue status/blocked Unable to be worked further until needs are met P2 Medium: Good to have, but can wait until someone steps up difficulty:moderate labels Oct 11, 2018
@lanzafame
Copy link
Contributor

lanzafame commented Oct 11, 2018 via email

@koalalorenzo
Copy link

koalalorenzo commented Oct 11, 2018

my 2 cents that maybe can help: echo is not really needed for what they have, sometimes using a framework is more painful than actually just integrating with the "lower level" api of go.

I am using a lot https://github.com/Sirupsen/logrus with some custom variables that are enabling different kind of logs and customisation to print to stdout vs stderr etc etc
What it allows me to do is to tag the different logs so I can differentiate in the analytics tools what kind of source it is (ex: api, libp2p, others)

As I am an heavy user of docker, I suggest to still use stdout instead of dumping logs in a file. Or make it optional to dump them somewhere. :D

@hsanjuan
Copy link
Collaborator Author

echo makes it super easy to add logging and some other things that are useful in APIs (auth, cors etc), but I'm happy not using it too.

@hsanjuan hsanjuan added this to the Q1 2019 Easy milestone Dec 19, 2018
@kishansagathiya
Copy link
Contributor

what is this issue blocked on?

@hsanjuan
Copy link
Collaborator Author

discussion

@hsanjuan hsanjuan removed the status/blocked Unable to be worked further until needs are met label Jul 23, 2019
@hsanjuan
Copy link
Collaborator Author

Actionables here:

  • A PR doing HTTP request logging for every request. By default printing them as DEBUG in the normal logs. A configuration option allows to specify a file-name to print those to a file.

@hsanjuan hsanjuan added exp/wizard Extensive knowledge (implications, ramifications) required and removed difficulty:moderate labels Jul 23, 2019
@kishansagathiya
Copy link
Contributor

kishansagathiya commented Aug 21, 2019

https://httpd.apache.org/docs/1.3/logs.html seems to be a good resource about how to do this.

@kishansagathiya
Copy link
Contributor

things that we would want to log loglevel(error, access), host, time, method, path, protocol, status, error, response size

@hsanjuan
Copy link
Collaborator Author

@hsanjuan
Copy link
Collaborator Author

kishansagathiya pushed a commit that referenced this issue Aug 21, 2019
This commit introduces logging for Cluster HTTP APIs.
It adds a config element `send_logs_to_file`, which tells whether logs
should be saved in a file or shown in standard output.

Requests are logged as per Apache Common Log Format (CLF)
http://httpd.apache.org/docs/2.2/logs.html#common

Fixes #574
@hsanjuan hsanjuan mentioned this issue Sep 9, 2019
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
exp/wizard Extensive knowledge (implications, ramifications) required help wanted Seeking public contribution on this issue kind/enhancement A net-new feature or improvement to an existing feature P2 Medium: Good to have, but can wait until someone steps up
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants