Distance support for table service

[niemeier-PSI]

The "table" service now optionally provides distances next to driving durations.

Api-Changes:
QUERY: The table service accepts a new (optional) parameter
"output_components" to specify which data-components to output. The parameters value is a ';'-separated list of keywords. Currently the keywords "durations" and "distances" are supported with obvious meanings. In case the "output-components" parameter is not specified, the default value is "durations". This way the API is backward compatible.
RESPONSE: The "durations" output is provided only when requested as output component, the format is unchanged. If the output-component "distances" is requested, the response contains a "distances" field of the same format as "distances".

Code-Changes:

• Distances are passed through the entire extraction/contration process and are part of the input files for osrm-routed.
• The many-to-many router calculates distances alongside durations
• During PhantomNode creation, the required partial distances on the "first-leg"/"last-leg" are computed on the uncompressed geometry (analogous to the PhantomNode's forward_weight/-offset)

Status:
The current version is fully functional and already in use in a customized version. It is marked as "not-ready", because updates to the API-documentation are still missing. I am waiting for your approval on the changes before
updating the documentation.

Use-Case:
The algorithms of our supply-chain optimization software PSIglobal
(http://www.psilogistics.com/en/solutions/supply-chain-management/) require pairwise distances for a huge number of points (for freight cost calculations the optimizations are based on). The excellent performance of OSRM would make it a perfect provider for the distance matrices. However, obtaining distances instead of durations is crucial for us. This is why we customized OSRM accordingly, and would be very happy if this customization would find its way into the main branch.

Memory-Concerns:
We are running this customization (more precisely, this customization rebased on tag v.5.2.7) on planet dataset. Memory consumption of osrm-routed is about 68,5 GB.
(Option --max-table-size 10000 was used)
Memory consumption on europe dataset is about 24 GB

[risinglf]

great job!!!!

[jellevictoor]

Awesome!

[RhinoDevel]

@niemeier-PSI Nice! Do you by any chance know the amount of HD memory needed for the unpatched planet and Europe datasets?

[niemeier-PSI]

hey guys, I'm glad you like it!

@RhinoDevel: I will check the memory requirements of v5.2.7 to be able to compare. Will take some time though as i first have to do the extraction/contraction for the unpatched version.

[RhinoDevel]

@niemeier-PSI Thanks a lot!! Although I did not mean to put more work on your plate - just thought you might already know the numbers, anyway.

[niemeier-PSI]

@RhinoDevel no, it makes sense to have these numbers! Anyway, it's not much work for me, only for my computer ;)

[niemeier-PSI]

Ok, so here is the memory consumption on my system (--max-table-size 10000) for
tag v5.2.7
planet: 63.9 GB
europe: 21.5 GB

As posted above, the memory consumption for the table-distances branch rebased on tag v5.2.7 is
planet: 68.5 GB
europe: 24 GB

All measurements performed after performing a quadratic table query involving 2500 locations in europe.
So we see a moderate increase in memory consumption, but (imho) nothing too drastic.

[ZupoLlask]

Congratulations! :-)

[TheMarex]

Thanks for making the effort of moving this upstream @niemeier-PSI! Clearly you understand the code base well. Great contribution! 👍 Sadly we can't accept this PR for the same reasons that we didn't accept previous contributions: It increases memory usage unconditionally, if you use the feature or not. I know this is however useful for a lot of people using OSRM for distance table computations, as such this is of course a very valuable contribution even if it does not live in the mainline.

There are two approaches I can see to implement distances in table queries:

1. Allow abitrary edge weights #2399 will allow arbitrary edge weights for routing. That means a distance based car profile becomes easy to implement. The table plugin could return two matrices: {weight_name} and durations. weight_name would be a string you specify in the profile, for example distance. This will have about the same memory increase that you are seeing right now, with the added benefit that it won't use any more memory if you don't use it.
2. Resurrect https://github.com/Project-OSRM/osrm-backend/tree/feature/general_manytomany. This approach is computationally more expensive then your current implementation but does not increase memory consumption at all.

In terms of API design we could try to adapt your solution so it is compatible with the API that would result from #2399 making it easy to switch to an OSRM 6.0 based implementation. For this the following logic would need to be implemented:

• No conditional flag for distances. The response should always include both like for example:

    {
         "distances": [[0, 2], [2, 0]],
         "durations": [[0, 0.1], [0.1, 0]]
    }

  We try very hard to limit the sprawl if API parameters. In this case having the flags is merely an optimization on computational time.

• No version bump as the above API is completly backwards compatible.

[jellevictoor]

What is not clear to me is why this can't be a build parameter? This way it
can remain in the master and based on the type of build you did, the
distances are included in the table.

Do I understand this correctly, by moving it to #2399 you 'misuse' the
weight to save the distance? This way the weight couldn't be used for other
purposes like discourage roads without influencing their speed.

[TheMarex]

What is not clear to me is why this can't be a build parameter?

If you see how many parts of the code this touches, you will recognize that adding a build parameter will do nothing less then making an already hard to maintain code base even harder to maintain.

The goal of #2399 is to return accurate duration values independent on what you use for routing. I see this an intended use.

[jellevictoor]

Hey TheMarex, thanks for you answer. The first point is clear to me, if the code is touched in to many ways, i understand that adding this will just make it more complex to maintain.
The second isn't completely clear. you would enter then distances as a ways weight? You say its to return accurate durations, how does distance come into play?

[niemeier-PSI]

@TheMarex : That is very sad.
I totally understand your concern about increasing memory consumption unconditionally for features that not everyone needs. I am not happy about this either, but I did not see good way around that did not involve a lot of changes in the program architecture (something that I clearly do not want to do without your consent).

But please be so fair to admit that by adding the lane guidance support feature in 5.3, you where in a similar situation and decided otherwise. I do not need the lande guidance feature, however it is activated unconditionally and increases the memory consumption in more or less the same way as this branch does.

I believe that simply rejecting a features just because it is not needed by everyone is not the best way to go. Instead one should think about changing the program architecture so that optional features can be activated/deactivated more easily (e.g. with command line parameters).
I would be willing to support on this, but you would have to express your willingness to go in that direction.

Unfortunately both of your suggestions are are not suitable for us:

1. If I understand Allow abitrary edge weights #2399 correctly, it would calculate distance-wise shortest route. This is not what we need. For our use-case we need the distances of fastest routes.
   Also with Allow abitrary edge weights #2399 the memory consumption would double up, as I would have to run two osrm services in parallel (with 63.9 GB each). This branch provides a solution with memeory increase of only 6%.
2. Also the approach from https://github.com/Project-OSRM/osrm-backend/tree/feature/general_manytomany is not suitable for us, as it significantly increases computation time. I am very willing to compromise on memory, but computation time is crucial and actually the strongest selling point of your router!

About your remark on the API: The reason why we made the output parameters optional is to keep the network traffic low: We do not want to increase the network traffic unconditionally , no matter if you use a feature or not. Actually, as our caller is a JAVA client, we seriously struggle parsing large JSON responses. Due to technical Java-VM limitations, the JSON response must not exceed 2GB

[TheMarex]

But please be so fair to admit that by adding the lane guidance support feature in 5.3, you where in a similar situation and decided otherwise. I do not need the lane guidance feature, however it is activated unconditionally and increases the memory consumption in more or less the same way as this branch does.

That is a fair comparison. Indeed this is the unfortunate result for prioritization our own business needs. Honestly I would really like to change that, lane guidance for pedestrians does not make sense, but it is in there. I don't have a good solution here yet and it has not been a priority for us.

Instead one should think about changing the program architecture so that optional features can be activated/deactivated more easily (e.g. with command line parameters)

I'm open for ideas here. Apart from clustering the code base with compile-time switches I don't see a way to do this. Maybe some template programming would be the answer here? We basically have a similar problem for #2477. As you already experienced features like this are heavily intervened with core data structures. Part of the reason OSRM is so fast is that it sacrificed flexibility for speed. (to be fair some of the inflexibility is also the result of an organically growing code base)

we seriously struggle parsing large JSON responses. Due to technical Java-VM limitations, the JSON response must not exceed 2GB

Have you investigated using JNI? I think @danpat once experimented with writing bindings for libosrm. With big queries I usually advice to skip the HTTP API, as serialization and deserialization can add quite some overhead as you are experiencing.

as it significantly increases computation time.

I'm curious did you benchmark that? I would be very interested in some number, as I never got around to investigate this myself.

[niemeier-PSI]

@TheMarex Thank you for your reply.
There are three separate discussion lines (Changes in the architecture to be more "optional feature friendly", alternatives for distance computation and API-issues. I will try to keep them separate to keep the post readable.

About the first one: Changes in the architecture to be more "optional feature friendly".

I agree that littering the code base with compile-time switches is not a good idea.
My idea is to introduce some payload data objects and pass them through the system
"generically". The payload objects itself can then be defined more flexible. Here even compiler-time switches would be an option because they would appear only locally in one place.
Here are some more details on that idea. It still is not fully fleshed out, and maybe there are still some hurdles to overcome that I am not aware of yet, but I hope you get the idea.

preprocessing steps (extraction mostly)

• Separate the core-data from the "payload". Right now, NodeBasedEdge contains lots of fields that essentially are just passed through (lane info, road classification, my new DistanceData object)
  Assume we put all this into a separate "payload struct" (e.g. called MetaData, ExtraData, PayloadData), overload operators where needed, and provide methods to perform preprocessing operations (e.g. a compress call for the the GraphCompressor, a call to write data to the disk). Then we can just pass this struct through the preprocessing. It still needs to be initialized in the ExctractionContainers, though. Any changes like adding additional meta data (e.g. info on toll distance is on my list) would require much less code changes, as the passing through of the payload object throught the different phases remains unchanged. Also, while not being a fan of littering the code with compiler-based switches, one could use them for the definition of this particular struct to selectively add/remove features at compile-time)
• At some point, the payload data should get serialized into a separate file. It probably is a good idea to follow your approach for the lance guidance data to not add them to the core graph structure, but keep them separate. One should decide on whether to have all that data in one file, or a separate file for each feature. I have no strong opinion either way
• Additionally, we need a different payload data structure that maintains payload data that must be maintained for the contracted graph.

router

• All payload data can be accessed via the datafacade (as you already implemented for the lane guidance info)
• The user should have the option to specify via command line parameters which services and features he intends to use.
• From these service/feature requests, the router must figure out which input data is required (Maybe this can be standardized in some way. E.g. the services get a new method that receives a list of features that are required, and returns a list of data elements that are needed.
• The datafacade must support selective loading of data
• Probably there is a huge memory saving potential here, e.g. if I only want to use the table service, I do not need to load street names etc.
• The many-to-many router should get some code to generically maintain payload data

For my distances feature, it would mean that I would remove the distance data from the EdgeBasedGraph and the QueryEdge, and instead would make this accessible via the datafacade

Second topic: alternatives for distance computation

as it significantly increases computation time.

I'm curious did you benchmark that? I would be very interested in some number, as I never got around to investigate this myself.

I did not. There was a discussion on this approach in #1353
where @systemed mentioned that the processing time for a size 33 square matrix bumps from 0.164s to 1.177s.
But taking a closer look, these numbers where for @systemed implementation of that approach (which is not publicly available), while you where referencing a different one by @DennisOSRM
Ok, maybe I will benchmark that one, although I would be surprised to not see a severe performance impact. You have to have a bit of patience though as I am away from office without proper access to our infrastructure for the next 3 weeks.

Third topic: API-issues

Have you investigated using JNI? I think @danpat once experimented with writing bindings for libosrm. With big queries I usually advice to skip the HTTP API, as serialization and deserialization can add quite some overhead as you are experiencing.

The problem is that in our architecture, the OSRM router is usually on a different machine than our software making the table request. For that reason, we are quite happy with the fact that you provide a fully functional webservice. Of course we could write another network service (using osrm as a library) running a proprietary protocol to overcome the limitations, but that would be one more component to maintain and it feels kind of redundant. On my shortlist on future contributions is the following approach: Add the option to respond with multipart HTTP responses. The JSON response would stay as before with one exception: Instead of having the table data as part of the JSON string, it would be transmitted binary in a binary part section of the HTTP response.

[dskrvk]

What's the status here?

[TheMarex]

You can get something similar if you change the weight_name in car.lua to distance. But then you also route on distance.

[oxidase]

@TheMarex many-to-many routing is based on durations, so in profile segment durations must be changed to distances. It is a little bit weired, but I think such values later must be completely abstracted from their names and use one value as a heap key and some other values as accumulated annotations if needed

[niemeier-PSI]

@dskrvk: Unfortunately, the status is unchanged, as there has been no response from the devs to my suggestions from August.

I will use this oppourtunity to summarize what happened so far, and to raise questions (to the devs) on how to go on from this.

Let me summarize what happened so far:

• I started a merge request for an additional feature, the option to get distances next to durations from the table service (Let me stress again that this is about the distances for duration-optimized routes, NOT shortest-distance routes!)
• @TheMarex reviewed and rejected it. Altough the quality was deemed good, the (legitimate) reason for rejection was an unconditional increase in memory consumption (by about 7%)
• I agreed with that concern, but pointed out that there is no satisfactory way to address this with the given architecture.
  I suggested to change the architecture to be more friendly to additional features that can be turned on and off at the users will (losslessly).
  I offered my support for designing and implementing those changes, provided that you are willing to go in that direction.
• @TheMarex asked for a suggestion on the architecture changes.
• I offered a suggestion in August
• Since then, there was no reponse to this.

I am now wondering on how to move on.
The only way to cope with @TheMarex s concern is to make changes to the architecure, which is something I cannot do without the devs consent if I ever want to see this merge request accepted.
But I get no feedback on the suggestions.

We plan to add additional features in the near future, which will build on this feature. This will make the diff even more complex and it will very quickly get to a point where a merge will get impossible, resulting in a permanent fork of OSRM for applications in transportation logistics.

Imho it would be very sad if this is what would happen. You guys did an outstanding job with osrm, and I would rather like to contribute to help it become THE general purpose OSRM routing engine, than creating yet another variant for some niche applications.

So my question to the devs is: What are your goals with the osrm project?
Is it intended to be THE OSM routing engine, or is it itended to be a routing engine tailored to specific needs?

If your answer tends towards the former, you have to provide an adequate architecture that can support optional features at no extra cost when turned off. It must be possible to integrate feature that are important to others, even if they are of no benefit to you. Otherwise OSRM cannot be THE osrm routing engine. As I already expressed repeatedly, I am happy to offer support on this.

[danpat]

We've had to pay the additional memory price for the implementation of #2399.

I think if we can do the following:

1. osrm-extract --output-edge-distances (outputs a .edge_distances file, defaults to not doing this)
2. Running osrm-routed looks for the optional file .edge_distances and loads it if available
3. Add a new API version of the table plugin /table/v2/.... that returns a new JSON structure for the API response, and has null for distance values if the data isn't available.

then we can implement this feature. I'm still hesitant to make it non-optional, but it's something we should have, this request comes up often.

[niemeier-PSI]

@danpat: I am very happy to hear this! Having the edge distances in a separate file and making it optional is a good idea. I will implement your suggestions and hope to see this feature merged afterwards.

Then I can finally start working on the next items on our "wishlist". :)

[frodrigo]

With the recent advances of min weight routing in place of duration come the need of getting duration and distance.
The table now returns weight, but for transportation logistics computation we need duration and distance matrix in addition of weight.

[developerbhuv007]

@damu4 yes still the same response

{
message: "Query string malformed close to position 93",
code: "InvalidQuery",
}

[sandeepgadhwal]

@developerbhuv007 did you remove the previous make of osrm-routed before intalling this version https://github.com/niemeier-PSI/osrm-backend/tree/v5.2.7-psi you can remove by going into the build folder of previous install and typing make uninstall i had the same problem, i removed all previous installs and did a clean install of this version and now it works without any problem example of url http://localhost:5000/table/v1/distance/-118.241483,34.072504;-118.329545,34.092053?sources=0&output_components=distances
results
{"code":"Ok","distances":[[0,9539.382812]],"sources":[{"hint":"YTAHgKScCIBinQAA6gAAABsBAABpAAAACAAAAHxIkkOGxXtDLqgEAKAsAwA2AgAAzsHz-HjfBwI1x_P4uOcHAgMAAQEZh_uz","location":[-118.242866,34.070392],"name":"Lilac Terrace"}],"destinations":[{"hint":"YTAHgKScCIBinQAA6gAAABsBAABpAAAACAAAAHxIkkOGxXtDLqgEAKAsAwA2AgAAzsHz-HjfBwI1x_P4uOcHAgMAAQEZh_uz","location":[-118.242866,34.070392],"name":"Lilac Terrace"},{"hint":"uJcLgLgnEoAxDgAAOAAAADAAAAAJAAAAAAAAAOCsYEJwVydCjMMGAI_DBgA2AgAA4G3y-BQ0CAI3b_L4FTQIAgEAAQEZh_uz","location":[-118.329888,34.092052],"name":"Cole Avenue"}]}

below command removes all the files related to osrm then you do a fresh install

cd / cd usr cd local cd bin sudo rm osrm-components cd .. cd include rm -R osrm sudo rm -R osrm cd .. cd bin sudo rm osrm-extract sudo rm osrm-parition sudo rm osrm-partition Sudo rm osrm-customize Sudo rm osrm-contract Sudo rm osrm-datastore Sudo rm osrm-routed sudo rm osrm-partition sudo rm osrm-customize sudo rm osrm-contract sudo rm osrm-datastore sudo rm osrm-routed cd .. cd lin sudo rm /usr/local/lib/libosrm.a sudo rm /usr/local/lib/libosrm_extract.a sudo rm /usr/local/lib/libosrm_partition.a sudo rm /usr/local/lib/libosrm_customize.a sudo rm /usr/local/lib/libosrm_update.a sudo rm /usr/local/lib/libosrm_contract.a sudo rm /usr/local/lib/libosrm_store.a sudo rm /usr/local/share/osrm/profiles sudo rm /usr/local/share/osrm/profiles/bicycle.lua sudo rm /usr/local/share/osrm/profiles/debug_example.lua sudo rm /usr/local/share/osrm/profiles/rasterbotinterp.lua sudo rm /usr/local/share/osrm/profiles/examples sudo rm /usr/local/share/osrm/profiles/examples/postgis.lua sudo rm /usr/local/share/osrm/profiles/foot.lua sudo rm /usr/local/share/osrm/profiles/testbot.lua sudo rm /usr/local/share/osrm/profiles/turnbot.lua sudo rm /usr/local/share/osrm/profiles/car.lua sudo rm /usr/local/share/osrm/profiles/lib sudo rm /usr/local/share/osrm/profiles/lib/pprint.lua sudo rm /usr/local/share/osrm/profiles/lib/relations.lua sudo rm /usr/local/share/osrm/profiles/lib/destination.lua sudo rm /usr/local/share/osrm/profiles/lib/utils.lua sudo rm /usr/local/share/osrm/profiles/lib/sequence.lua sudo rm /usr/local/share/osrm/profiles/lib/set.lua sudo rm /usr/local/share/osrm/profiles/lib/access.lua sudo rm /usr/local/share/osrm/profiles/lib/profile_debugger.lua sudo rm /usr/local/share/osrm/profiles/lib/guidance.lua sudo rm /usr/local/share/osrm/profiles/lib/way_handlers.lua sudo rm /usr/local/share/osrm/profiles/lib/maxspeed.lua sudo rm /usr/local/share/osrm/profiles/lib/tags.lua sudo rm /usr/local/share/osrm/profiles/test.lua sudo rm /usr/local/share/osrm/profiles/rasterbot.lua sudo rm /usr/local/lib/pkgconfig/libosrm.pc

[developerbhuv007]

@sandeepgadhwal Thanks for the reply. I removed the whole build directory before installing the new version.

[sandeepgadhwal]

link to repository forked by me : https://github.com/sandeepgadhwal/osrm-backend-distance it is the exact checked out version for distance tables.

[developerbhuv007]

@sandeepgadhwal Thanks a lot. It is working fine now.

Now I am getting following response when I am trying to find the matrix for more number of geo points.

{
message: "Too many table coordinates",
code: "TooBig",
}

Thanks in Advance

[sashakh]

Use osrn-routed with '--max-table-size' option to increase the limit.

…

On Dec 14, 2017 10:42 AM, "Bhuvnesh" ***@***.***> wrote: @sandeepgadhwal <https://github.com/sandeepgadhwal> Thanks a lot. It is working fine now. Now I am getting following response when I am trying to find the matrix for more number of geo points. { message: "Too many table coordinates", code: "TooBig", } Thanks in Advance — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#2764 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABCIjAZWc0L36FQkS7El4gk197O3nt1Dks5tAN9wgaJpZM4Ji5at> .

[sandeepgadhwal]

@developerbhuv007 was your problem solved by using --max-table-size as suggested by @sashakh

[developerbhuv007]

@sandeepgadhwal yes it solved my problem.

Thanks a lot @sashakh

[sandeepgadhwal]

Is there a way to bring down the response size from

{"code":"Ok","distances":[[0,9539.382812]],"sources":[{"hint":"YTAHgKScCIBinQAA6gAAABsBAABpAAAACAAAAHxIkkOGxXtDLqgEAKAsAwA2AgAAzsHz-HjfBwI1x_P4uOcHAgMAAQEZh_uz","location":[-118.242866,34.070392],"name":"Lilac Terrace"}],"destinations":[{"hint":"YTAHgKScCIBinQAA6gAAABsBAABpAAAACAAAAHxIkkOGxXtDLqgEAKAsAwA2AgAAzsHz-HjfBwI1x_P4uOcHAgMAAQEZh_uz","location":[-118.242866,34.070392],"name":"Lilac Terrace"},{"hint":"uJcLgLgnEoAxDgAAOAAAADAAAAAJAAAAAAAAAOCsYEJwVydCjMMGAI_DBgA2AgAA4G3y-BQ0CAI3b_L4FTQIAgEAAQEZh_uz","location":[-118.329888,34.092052],"name":"Cole Avenue"}]}

to

{"code":"Ok","distances":[[0,9539.382812]]}

query what i am doing is http://localhost:5000/table/v1/distance/-118.241483,34.072504;-118.329545,34.092053?sources=0&output_components=distances

[niemeier-PSI]

@sandeepgadhwal Not with version 5.2.7. If I understand the API-Docs of vanilla ORSRM correctly (http://project-osrm.org/docs/v5.10.0/api/#general-options), using the option "generate_hints=false" would do the trick. But AFAIK the option is not supported in 5.2.7.

[sandeepgadhwal]

@niemeier-PSI why they did not include such a wonderful feature in the official version, thanks for your effort and great contribution i hope they will include it in future as more and more people are interested in it.

[niemeier-PSI]

@sandeepgadhwal Thank you :)
Yes, lets not give up hope. Maybe one day...

[hopewise]

As for my case, I just need durations , thanks @niemeier-PSI for mentioning the option "generate_hints=false" , but what about removing destinations and sources ?

[niemeier-PSI]

@hopewise AFAIK it is not possible to suppress the sources and destinations fields.
Anyway I recommend to not just ignore these source/destination field. The coordinates returned in those fields are the snapping coordinates (i.e. coordinates on the road network closest to your input coordinates). The returned durations/distances are w.r.t. these snapping coodinates. For productive use, you should validate that snapping coordinates and real coordinates are "close enough".

[Migaelh]

Good day.

I am trying to setup the OSRM that includes the distances and durations. It seems that i am unable to get the distances and duration in the table request. I have tried the following:

1. Followed @niemeier-PSI intstructions at 24 March 2017
2. Build https://github.com/niemeier-PSI/osrm-backend/tree/v5.2.7-psi with the following commands:

• mkdir -p build
• cmake .. -DPAYLOAD_TYPE=DurationsAndDistances
  ( this step CMake returned a warning "CMake Warning:
  Manually-specified variables were not used by the PAYLOAD_TYPE" )
• cmake --build .
• sudo cmake --build . --target install

3. After the build I ran

• osrm-extract map.osm.pbf -p ../profiles/car.lua
  returned an error "TBB Warning: Exact exception propagation is requested by application but the linked library is built without support for it
  terminate called after throwing an instance of 'tbb::captured_exception'
  what(): lua: error: lua: error: ../profiles/car.lua:386: sol: attempt to index (set) nil value "is_access_restricted" on userdata (bad (misspelled?) key name or does not exist)
  "
  when this failed I used the car.lua from the master tag and it worked then
• osrm-contract map.osrm
• osrm-routed map.osrm

4. The request:
   http://{own-server_ip}/table/v1/driving/77.0697,28.499;77.0497,28.459;77.0397,28.469;77.0797,28.519;77.0997,28.559?output_components=distances;durations
   response:
   Reponse:
   {
   "message": "Query string malformed close to position 93",
   "code": "InvalidQuery"
   }

I also tried @sandeepgadhwal's post on Dec 14, 2017 and still got the same results.

Am i missing something or doing something wrong? I have tried different versions and build for a few days now and still not getting the distances and durations back.

[grudolf]

The correct branch is table-distances.

[Migaelh]

Thanks you! It's woriking now

…

On 14 Jan 2018 13:26, "Gregor Rudolf" ***@***.***> wrote: The correct branch is table-distances. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#2764 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AfSlYs4Vd-lT0PvHpTAEZJDT1x9MsRKgks5tKeRSgaJpZM4Ji5at> .

[RhinoDevel]

Just sayin'..

Top issue:

Top pull request:

[sandeepgadhwal]

Don't know why this is not available in official version .

…

On 16-Jan-2018 1:07 PM, "RhinoDevel" ***@***.***> wrote: Just sayin'.. [image: 1] <https://user-images.githubusercontent.com/5845035/34976735-3346d750-fa98-11e7-92e3-d794760c390d.png> [image: 2] <https://user-images.githubusercontent.com/5845035/34976746-3807c678-fa98-11e7-8c3c-fe69222b42cd.png> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2764 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ARpk2J0XKBKjb3aJDlGHVQVILq3hsKq2ks5tLFGjgaJpZM4Ji5at> .

[pango89]

Hello,
I have cloned the "table-distances" branch and followed the instructions as mentioned. I built by using the "-DPAYLOAD_TYPE = DurationsAndDistances" option.
With query that uses v2 as version, I am getting: "Service table not found!", "Code": "InvalidVersion".

With query that uses v1 as version, I am getting: { message : "Query string malformed close to position 93", code : "InvalidQuery"}
Can someone please tell me where i am doing it wrong?

[jellevictoor]

any plans to merge this with the 5.12 version where the 'avoid' feature is an option?

[jgabriels]

I would also like to vote for this feature to be merged into main.
I also would honestly like to know what the use case even is for a table with durations but no distances.
To me, if you were going to include only one, only returning distances makes a lot more sense.

[jgabriels]

For anyone else who is wondering, this is still working great in 2018:
git clone -b v5.2.7-psi https://github.com/niemeier-PSI/osrm-backend.git
with the following build instructions:
https://github.com/niemeier-PSI/osrm-backend#quick-start
I used the following dataset:
http://download.geofabrik.de/north-america-latest.osm.pbf
on a 16GB digitalocean instance with 100GB of swap and swappiness of 20
and it barely touched the swap:

Mem: 16432232 15017332 709708 2284 705192 1128032
Swap: 104857596 1137892 103719704

The whole process took less than 12 hours to build and call time is excellent:

time lynx --source 'http://127.0.0.1:5000/table/v1/driving/-117.1611,32.7157;-74.24,41.80;-78.51,43.02?sources=0&output_components=distances'|python -m json.tool >data

0.01user 0.00system 0:00.07elapsed 15%CPU (0avgtext+0avgdata 7188maxresident)k
0inputs+0outputs (0major+744minor)pagefaults 0swaps

Even with 100 records response time is still under half a second:

0.01user 0.00system 0:00.20elapsed 5%CPU (0avgtext+0avgdata 7160maxresident)k
0inputs+0outputs (0major+756minor)pagefaults 0swaps

[igormcoelho]

This thread is hopeless... everybody else is already using niemeier-psi solution, and the admins here don't have the minimum sense of understanding the importance of all this... Congratulations again @niemeier-PSI , I have already cited you in two academic papers for your valuable (and not recognized here) contribution.

[danpat]

For everyone else following this ticket, there is active work on this feature over in these two PRs:

Compute durations at runtime: #4876
Distances in the matrix: #4990

As stated earlier, we will not merge this PR. Once those two PRs are merged, we will probably close this one.

The main blocker here was the additional memory usage, and dev bandwidth - @niemeier-PSI I'm sorry we didn't have the time/focus to bring this into the mainline after you made your changes in #2764 (comment).

The two PR's above will implement distance response like so:

1. Remove duration properties from the routing graph, which will save quite a bit of memory
2. Unpack paths during matrix calculation to re-derive total durations. This will slow things down a bit.
3. Implement an LRU path-unpacking cache that will alleviate most of the slowdown of (2), and use less memory than was saved in (1)
4. Do the same thing for distance calculations (calculate at runtime through path unpacking with a cache)

The approach looks promising - performance is only slightly impacted, overall memory consumption is reduced, and distances will become available in the response, working with all current features in OSRM (excludes/avoid, CH & MLD, etc).

[TheMarex]

This is now available in OSRM 5.18 using the query options &annotations=duation,distance for the table plugin.

[danpat]

This pre-calculation (i.e. fast) functionality has been re-implemented in OSRM 5.20.0 as part of #5251