File ignored if mtime before ctime with 2.2 but not 2.1

[jimmyjones2]

LS config:

input {
  file {
    path => "/home/jimmy/logstash-testing/sample.log"
    start_position => "beginning"
    sincedb_path => "/dev/null"
  }
}
output {
  stdout {
    codec => rubydebug
  }
}

Create a file sample.log containing "hello". Then run

[jimmy@localhost logstash-testing]$ logstash-2.1.1/bin/logstash -f sample.conf 
Settings: Default filter workers: 2
Logstash startup completed
{
       "message" => "hello",
      "@version" => "1",
    "@timestamp" => "2016-02-19T15:46:22.231Z",
          "host" => "localhost.localdomain",
          "path" => "/home/jimmy/logstash-testing/sample.log"
}
^CSIGINT received. Shutting down the pipeline. {:level=>:warn}
Logstash shutdown completed

[jimmy@localhost logstash-testing]$ logstash-2.2.2/bin/logstash -f sample.conf 
Settings: Default pipeline workers: 4
Logstash startup completed
{
       "message" => "hello",
      "@version" => "1",
    "@timestamp" => "2016-02-19T15:46:41.609Z",
          "path" => "/home/jimmy/logstash-testing/sample.log",
          "host" => "localhost.localdomain"
}
^CSIGINT received. Shutting down the pipeline. {:level=>:warn}
Logstash shutdown completed

as expected works fine. Now change the mtime to be before the ctime and try again:

[jimmy@localhost logstash-testing]$ touch -m -t 201601010000 sample.log 
[jimmy@localhost logstash-testing]$ logstash-2.2.2/bin/logstash -f sample.conf 
Settings: Default pipeline workers: 4
Logstash startup completed
^CSIGINT received. Shutting down the pipeline. {:level=>:warn}
^[[A^[[DLogstash shutdown completed

[jimmy@localhost logstash-testing]$ logstash-2.1.1/bin/logstash -f sample.conf 
Settings: Default filter workers: 2
Logstash startup completed
{
       "message" => "hello",
      "@version" => "1",
    "@timestamp" => "2016-02-19T15:47:25.978Z",
          "host" => "localhost.localdomain",
          "path" => "/home/jimmy/logstash-testing/sample.log"
}
^CSIGINT received. Shutting down the pipeline. {:level=>:warn}
Logstash shutdown completed

Logstash 2.1 is happy with this, but 2.2 isn't...

[jimmyjones2]

@guyboertje Would you mind having a look please?

[guyboertje]

2.2 is using the ignore-older setting of 24 hours to skip the file. If you use the --debug switch you should see the log output detailing the skip.

[jimmyjones2]

Thanks @guyboertje. Can the release notes be update to warn about this as IMHO it's a non-obvious change? Currently says:

Added config option ignore_older which monitors if a file that was last modified before the specified timespan in seconds, then its contents are ignored

but nothing about the new default of 24 hours. The docs also don't make it clear at the top and even for the parameter it says:

If this option is specified, when the file input discovers a file that was last modified before the specified timespan in seconds, the file is ignored.

But I didn't specify the parameter and still got this behaviour.

[guyboertje]

I will update the docs - thanks for the heads up.