Heads up: ProGit2 build process is not providing .mobi files for now #1498

HonkingGoose · 2020-08-28T18:42:44Z

Which download is failing?

The link to the latest .mobi files at https://git-scm.com/book/en/v2 is gone.

Problem

The .mobi files are gone upstream, and so the link to the .mobi files is too.

Context

The ProGit2 project had to disable .mobi builds with progit/progit2#1497 because the kindlegen gem that we use to generate the .mobi files whenever Travis CI makes a build cannot be downloaded anymore. See progit/progit2#1496 for more on this.

Steps to Reproduce

Go to the https://git-scm.com/book/en/v2 once the git-scm site build is refreshed.
Click on the .mobi link.
It will probably break.

Other details

Hi git-scm team, I wanted to provide a heads up for you. I hope I filled the correct issue template! 😄

The text was updated successfully, but these errors were encountered:

pedrorijo91 · 2020-08-28T19:06:51Z

wow :) thanks a lot for the heads-up @HonkingGoose !

is there any ETA for getting the .mobi files working again? If I understood correctly, amazon stopped serving a binary which kindlegen wrapped?

pedrorijo91 · 2020-08-28T19:09:53Z

I think it's just a matter of removing from the UI until things are solved, in these lines

git-scm.com/app/views/books/show.html.erb

Lines 26 to 30 in 8b4de72

    
           <div class="ebooks"> 
        
             <% if @book.ebook_mobi %> 
        
               <a href="<%= @book.ebook_mobi %>"><img width="50px" src="/images/mobi.png"/></a> 
        
             <% end %> 
        
           </div>

HonkingGoose · 2020-08-28T19:14:21Z

I cannot provide any ETA on when the .mobi files will be available again, because I'm not the one that has the ability or knowledge to fix this.

You can follow the asciidoctor-epub3 issue here: asciidoctor/asciidoctor-epub3#363
and the upstream kindlegen issue here: tdtds/kindlegen#42

EDIT:

If I understood correctly, amazon stopped serving a binary which kindlegen wrapped?

Yes that's right. The replacement that Amazon provides (Kindle Previewer) does not have a Linux binary.

HonkingGoose · 2020-08-28T19:22:22Z

I think it's just a matter of removing from the UI until things are solved, in these lines

git-scm.com/app/views/books/show.html.erb

Lines 26 to 30 in 8b4de72

<div class="ebooks">

<% if @book.ebook_mobi %>

<a href="<%= @book.ebook_mobi %>"><img width="50px" src="/images/mobi.png"/></a>

<% end %>

</div>

Feel free to propose a pull request to delete the .mobi link. I would recommend putting a comment in, so that it's clear why the link was removed, and maybe link to this issue and the upstream ProGit2 issue.

That way it's documented in your code what happened and why, and people can check if/when the issue is fixed and put the .mobi link back.

HonkingGoose · 2020-08-29T10:51:48Z

I visited the page after the site build refreshed: the .mobi link is gone. The pdf and epub have links to version 2.1.259, which are current as of now.

So maybe this takes no action on your side, and the system will just pick up the .mobi files once they are available again. 😄

jnavila · 2020-08-29T11:02:59Z

Calibre has a converter to azw, but I fear the quality of the output is not the same as kindlegen.

HonkingGoose · 2020-08-29T11:23:08Z

Yeah I searched for some alternative tool-chain and stumbled into Calibre as well, it does have a command line interface that we maybe could use. That would probably require another Travis CI build chain to be built.

And we would first need to know if the .epub -> Calibre -> Kindle output is any good. So maybe we can get a Kindle user to do the conversion and report back on the results?

peff · 2020-09-01T00:25:22Z

So maybe this takes no action on your side, and the system will just pick up the .mobi files once they are available again.

Yeah, I think that's what's supposed to (and will) happen. But thank you very much for letting us know about the situation.

I suppose we could also provide an older version of the .mobi, if somebody feels strongly enough about that format that they'd prefer out-of-date content to using .epub or whatever. But I'd prefer not to get into that unless somebody asks for it.

HonkingGoose · 2020-09-09T19:27:36Z

Yeah, I agree: only provide outdated .mobi files if there is actual demand.

This is also a good way to figure out if those .mobi builds are actually valued at all... I've yet to see a issue report here, or on the ProGit2 repo asking about the .mobi files.

pedrorijo91 · 2020-09-09T20:31:19Z

not sure if it's possible to find out how many accesses to the mobi file there were before this. The url seems to be something like

https://github.com/progit/progit2/releases/download/2.*.*/progit.pdf

I can try to find out during the weekend perharps

HonkingGoose · 2020-09-10T07:39:09Z

Having some historical data would help with making the judgement call if the .mobi files are even worth the support/maintenance cost.

I've seen no comments/issues on the ProGit2 repo, asking about the .mobi files. So it could be that those .mobi files are not really in high demand.
Maybe people just use the version of the book hosted on the website (.html) or download the .epub and .pdf files for their e-book reader or local workstation.

HonkingGoose · 2021-06-15T15:28:42Z

I've opened a PR upstream at the ProGit2 repository that should re-enable the .mobi files. See progit/progit2#1655, once that's merged I'll close this issue.

pedrorijo91 · 2021-06-19T08:23:43Z

seems like the PR was already merged right @HonkingGoose ?

HonkingGoose · 2021-06-19T12:15:53Z

Yeah, but it's not working... I'm not sure what I need to do to fix it... 😕

HonkingGoose · 2021-06-29T18:24:34Z

ProGit2 is publishing .mobi files again! Go to https://github.com/progit/progit2/releases to see the progit.mobi files. 😄

The only thing left to do is to verify that git-scm picks up the new files correctly when your Heroku build process runs?

By the way, the progit/progit2 repository now uses a main branch, instead of a master branch. This shouldn't cause issues as @peff already did some preparatory work with PR #1502.

But you should still verify that things work properly after both those changes. 😉

HonkingGoose · 2021-06-29T18:28:54Z

Having some historical data would help with making the judgement call if the .mobi files are even worth the support/maintenance cost.

I've seen no comments/issues on the ProGit2 repo, asking about the .mobi files. So it could be that those .mobi files are not really in high demand.
Maybe people just use the version of the book hosted on the website (.html) or download the .epub and .pdf files for their e-book reader or local workstation.

The progit2 repository maintainer has said that they consider providing the .mobi files to be very important. So we'll probably continue to provide these files, unless we encounter new issues with the toolchain. 😄

peff · 2021-06-29T19:00:44Z

Hmm, now we don't seem to be generating links for any books! We find the appropriate assets for each release based on their content_type field. But that has changed recently. Running this ruby snippet:

require "octokit"
o = Octokit::Client.new(access_token: ENV["GITHUB_API_TOKEN"])
o.releases("progit/progit2").each do |rel|
        rel.assets.each do |asset|
                puts "#{rel.tag_name} #{asset.name} #{asset.content_type}"
        end
end

yields:

2.1.313 progit.epub raw
2.1.313 progit.html raw
2.1.313 progit.mobi raw
2.1.313 progit.pdf raw
2.1.312 progit.epub raw
2.1.312 progit.html raw
2.1.312 progit.pdf raw
2.1.311 progit.epub raw
2.1.311 progit.html raw
2.1.311 progit.pdf raw
2.1.310 progit.epub raw
2.1.310 progit.html raw
2.1.310 progit.pdf raw
2.1.309 progit.epub raw
2.1.309 progit.html raw
2.1.309 progit.pdf raw
2.1.308 progit.epub application/epub+zip
2.1.308 progit.html text/html
2.1.308 progit.pdf application/pdf
2.1.307 progit.epub application/epub+zip
2.1.307 progit.html text/html
2.1.307 progit.pdf application/pdf
2.1.306 progit.epub application/epub+zip
2.1.306 progit.html text/html
2.1.306 progit.pdf application/pdf
2.1.305 progit.epub application/epub+zip
2.1.305 progit.html text/html
2.1.305 progit.pdf application/pdf
[...and so on...]

So it looks like something changed recently on the release-generation side, and the content-types were lost. We probably could switch to matching the extension in the filename, but I suspect it would be worth fixing the content-type problem for other reasons (e.g., I imagine it impacts the content-type GitHub sends to users when they click on the releases in a browser).

HonkingGoose · 2021-06-29T19:04:50Z

We did just migrate ProGit2 from Travis CI -> GitHub Actions. Maybe that's breaking something?

EDIT:

Link to GitHub Action workflow directory: https://github.com/progit/progit2/tree/main/.github/workflows
Link to changed rakefile on ProGit2 repo: https://github.com/progit/progit2/blob/main/Rakefile

peff · 2021-06-29T19:05:27Z

So it looks like something changed recently on the release-generation side, and the content-types were lost.

Diffing 2.1.308 and 2.1.309, it looks like the release creation now uses https://github.com/ncipollo/release-action, but doesn't pass artifactContentType, so the default is raw. It's not clear to me if that property can take a comma-separated list like artifacts does.

peff · 2021-06-29T19:10:54Z

It's not clear to me if that property can take a comma-separated list like artifacts does.

Looks like no. The code seems to apply a single content-type to every artifact. So a fix would probably involve extending that action, and then updating progit2 to make use of the new feature. Yikes.

Even if that happens, it may take a while. I think we should switch to selecting based on the names in the meantime.

Since progit2 2.1.309 switched to using a GitHub Action to generate releases, it no longer sets the content-type field of the release assets. And thus we fail to find any ebooks to link, since we are looking for assets with "application/pdf", and so on. Instead, let's do a suffix match against the name of each asset, looking for the appropriate file extension. This works fine, since the names are all obvious ("progit.epub", and so on). And it should be backwards-compatible with older translations that haven't yet picked up the new Actions-based workflow, since they used the same sensible names. We may revert this later if the assets start generating with correct content-types again. But it looks to be non-trivial (there's some discussion in #1498), so this seems like a good solution in the meantime.

peff · 2021-06-29T19:21:16Z

Should be fixed by #1615. Running it locally restores the epub and pdf links, and now generates a mobi link, too. 😅

peff · 2021-06-29T19:42:03Z

OK, with #1615 merged, I kicked off a manual book generation job, purged the cloudflare cache, and now things are looking good: the main https://git-scm.com/book page has three links (epub, pdf, and mobi). So I think we're probably good to close this issue now!

HonkingGoose · 2021-06-29T19:48:37Z

Looks good to me on the deployed site as well.

I agree, let's close this issue! 😄

HonkingGoose mentioned this issue Sep 22, 2020

Discussion: does the ProGit2 project want to keep providing Kindle files? progit/progit2#1528

Closed

This comment has been minimized.

# to view

peff mentioned this issue Jun 29, 2021

book2: match ebook release assets by name, not content-type #1615

Merged

HonkingGoose closed this as completed Jun 29, 2021

HonkingGoose mentioned this issue Jun 30, 2021

GitHub Action release workflow doesn't provide contentType progit/progit2#1673

Open

This comment was marked as spam.

# to view

git deleted a comment from muhamadhafizy1991 Oct 7, 2021

git blocked muhamadhafizy1991 Oct 7, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Heads up: ProGit2 build process is not providing .mobi files for now #1498

Heads up: ProGit2 build process is not providing .mobi files for now #1498

HonkingGoose commented Aug 28, 2020 •

edited

Loading

pedrorijo91 commented Aug 28, 2020

pedrorijo91 commented Aug 28, 2020

HonkingGoose commented Aug 28, 2020 •

edited

Loading

HonkingGoose commented Aug 28, 2020

HonkingGoose commented Aug 29, 2020

jnavila commented Aug 29, 2020

HonkingGoose commented Aug 29, 2020

peff commented Sep 1, 2020

HonkingGoose commented Sep 9, 2020

pedrorijo91 commented Sep 9, 2020

HonkingGoose commented Sep 10, 2020

HonkingGoose commented Jun 15, 2021

pedrorijo91 commented Jun 19, 2021

HonkingGoose commented Jun 19, 2021

This comment has been minimized.

HonkingGoose commented Jun 29, 2021

HonkingGoose commented Jun 29, 2021

peff commented Jun 29, 2021

HonkingGoose commented Jun 29, 2021 •

edited

Loading

peff commented Jun 29, 2021

peff commented Jun 29, 2021

peff commented Jun 29, 2021

peff commented Jun 29, 2021

HonkingGoose commented Jun 29, 2021

This comment was marked as spam.

Heads up: ProGit2 build process is not providing .mobi files for now #1498

Heads up: ProGit2 build process is not providing .mobi files for now #1498

Comments

HonkingGoose commented Aug 28, 2020 • edited Loading

Which download is failing?

Problem

Context

Steps to Reproduce

Other details

pedrorijo91 commented Aug 28, 2020

pedrorijo91 commented Aug 28, 2020

HonkingGoose commented Aug 28, 2020 • edited Loading

HonkingGoose commented Aug 28, 2020

HonkingGoose commented Aug 29, 2020

jnavila commented Aug 29, 2020

HonkingGoose commented Aug 29, 2020

peff commented Sep 1, 2020

HonkingGoose commented Sep 9, 2020

pedrorijo91 commented Sep 9, 2020

HonkingGoose commented Sep 10, 2020

HonkingGoose commented Jun 15, 2021

pedrorijo91 commented Jun 19, 2021

HonkingGoose commented Jun 19, 2021

This comment has been minimized.

HonkingGoose commented Jun 29, 2021

HonkingGoose commented Jun 29, 2021

peff commented Jun 29, 2021

HonkingGoose commented Jun 29, 2021 • edited Loading

peff commented Jun 29, 2021

peff commented Jun 29, 2021

peff commented Jun 29, 2021

peff commented Jun 29, 2021

HonkingGoose commented Jun 29, 2021

This comment was marked as spam.

HonkingGoose commented Aug 28, 2020 •

edited

Loading

HonkingGoose commented Aug 28, 2020 •

edited

Loading

HonkingGoose commented Jun 29, 2021 •

edited

Loading