Buildpack not working

[yusjacobs]

Getting this when deploying in heroku:

Using buildpacks:
1. https://github.com/jonathanong/heroku-buildpack-ffmpeg-latest.git
2. heroku/python
-----> ffmpeg app detected
-----> Installing ffmpeg
Variable FFMPEG_DOWNLOAD_URL isn't set, using default value
Downloading https://johnvansickle.com/ffmpeg/builds/ffmpeg-git-amd64-static.tar.xz
Unable to download ffmpeg: 000

[stevenmaguire]

I am seeing the same thing and the ENV variable suggestion is also not working - and I think that is related to the two open PRs here.

[yusjacobs]

This started happening today. At the same time the scheduled tests started to fail

[knagode]

This is happening in our environment too.

I can download https://johnvansickle.com/ffmpeg/builds/ffmpeg-git-amd64-static.tar.xz without any issues. I wonder how to resolve this? Can we help? How?

[agieche]

Same here, started to fail today. Moving the buildpack to the top or bottom of the buildpack list did not help either. Setting FFMPEG_DOWNLOAD_URL is not helping, since it is somehow not picked up by the deployment.

[agieche]

I filed a ticket at heroku support. Will keep you in loop, if there is a suggestion from them.

[yusjacobs]

In the meantime i downloaded the binary from https://johnvansickle.com/ffmpeg/builds/ffmpeg-git-amd64-static.tar.xz and added it to my repository. I then deployed the app to Heroku and it is working. Just make sure you call the correct PATH to the binary in your code.

[knagode]

@yusjacobs

Just make sure you call the correct PATH to the binary in your code.

Did you simply put this file in public directory and then use:

FFMPEG_DOWNLOAD_URL

to download it? (In open PR-s I see that this doesn't work for some people)

[yusjacobs]

I created a bin folder in my repo and placed the binaries in there, both ffmpeg and ffmprobe. You can download both here: https://johnvansickle.com/ffmpeg/builds/ffmpeg-git-amd64-static.tar.xz

I then updated my commands like this:

command = ["./bin/ffmpeg", "-ss",.....

[ekaj2]

Same issue on my end

[mindtonic]

👋🏻

[sasharevzin]

I filed a ticket at heroku support. Will keep you in loop, if there is a suggestion from them.

We need someone with an enterprise account so they can escalate it quickly and not wait for two days for their reply 😭

[agieche]

Looked a little deeper into the issue. There seems to be a SSL TLS issue, when the heroku machine curls the tar.xz file, resultung in a status code 000, which is interpreted as error by the buildpack. However, when moving the archive to a different location, the buildpack works flawlessly again.

I forked the original buildpack and adjusted the url to a copy of the archive on my servers. Problem solved, deployment is working again.

If anyone wants to have a look on my workaround fork in order to build their own: https://github.com/agieche/heroku-buildpack-ffmpeg-latest

You can use it for now as drop in replacement of the current non working build pack until things have been sorted out:

heroku buildpacks:add --index 1 https://github.com/agieche/heroku-buildpack-ffmpeg-latest.git

Keep in mind, that I will remove the file in a resaonable amount of time after the original url is working again due to a future fix of heroku.

[stevenmaguire]

Thanks @agieche. To clarify, 1) your diagnosis is that johnvansickle.com SSL cert is causing the problem and 2) your fork is only pointing to a different remote URL where an SSL issue is not present, correct?

The thing that I am still not seeing function correctly is the FFMPEG_DOWNLOAD_URL env variable. It seems that if I am correct in both of those points, we should not need a new fork of the package, merely a successful path to use FFMPEG_DOWNLOAD_URL to point to a new remote URL.

[agieche]

Thanks @agieche. To clarify, 1) your diagnosis is that johnvansickle.com SSL cert is causing the problem and 2) your fork is only pointing to a different remote URL where an SSL issue is not present, correct?

The thing that I am still not seeing function correctly is the FFMPEG_DOWNLOAD_URL env variable. It seems that if I am correct in both of those points, we should not need a new fork of the package, merely a successful path to use FFMPEG_DOWNLOAD_URL to point to a new remote URL.

I tried using the env variable at first, but I could not find an obvious way to access the Heroku env vars from within the shell script. Otherwise, this fork would be unnecessary.

I don't think, it is an issue at the end of https://www.johnvansickle.com/ffmpeg/ since curling it from my local machine is working with a correct 200 status code. Seems more of an issue in the local networking of heroku, but I'm not a 100% on this.

[agieche]

Checked my forked buildpack on two different production apps and deployments went fine.

[ekaj2]

Just fixed it for my production app as well. Appreciate the quick hotfix @agieche

[agieche]

I added support for reading the ENV variables within my fork thanks to this PR: https://github.com/jonathanong/heroku-buildpack-ffmpeg-latest/pull/61/files

I set the ENV variable FFMPEG_DOWNLOAD_URL and restarted dynos and the variable was picked up correctly.

[stevenmaguire]

I set the ENV variable FFMPEG_DOWNLOAD_URL and restarted dynos and the variable was picked up correctly.

Did you have success setting this variable in the Heroku App (heroku config ... etc) or the app.json "env" key, or both?

[agieche]

I set the ENV variable FFMPEG_DOWNLOAD_URL and restarted dynos and the variable was picked up correctly.

Did you have success setting this variable in the Heroku App (heroku config ... etc) or the app.json "env" key, or both?

I added the code to my fork. Then set the env variable within web interface (https://dashboard.heroku.com/apps/my-app-16815/settings) and deployed from CLI. Then, the Env variable was picked up correctly.

[stevenmaguire]

I added the code to my fork. Then set the env variable within web interface (https://dashboard.heroku.com/apps/my-app-16815/settings) and deployed from CLI. Then, the Env variable was picked up correctly.

Awesome. Thanks!

[jharrilim]

I've taken a different approach to resolving this. Instead of using the precompiled binaries from johnvansickle.com, I've moved to using the official Ubuntu apt package.

This can be done by:

1. Adding the Heroku apt buildpack to your app
2. Creating an Aptfile in your repo with this as its contents:

ffmpeg

3. Removing this buildpack

[mmaloon]

With the Apt buildpack, when I re-deploy then try to run ffmpeg -version in the heroku shell, I receive an error with the messageerror while loading shared libraries.... The deploy also results in a much larger slug size (~250MB with no other buildpacks installed)

Another alternative buildpack is the heroku-buildpack-activestorage-preview which also installs the ffmpeg binary.

heroku buildpacks:add -i 1 https://github.com/heroku/heroku-buildpack-activestorage-preview

This buildpack also seems to fix the segfault issue on heroku-22 stack (#69)

[jonathanong]

Thanks @agieche, I added you as a collaborator. I don't use this build pack anymore. If Heroku wants to own this package, let me know!

[agieche]

Merged #61, so you can set a correct download url via env variable now. No definitive answer from heroku, but they are looking into it.

[diegotorpoco]

@agieche What value should the env variable FFMPEG_DOWNLOAD_URL in heroku client should take? many thanks!

[kayvaninvemo]

@agieche What value should the env variable FFMPEG_DOWNLOAD_URL in heroku client should take? many thanks!

Just add a new env variable in heroku->setting
FFMPEG_DOWNLOAD_URL : https://cookycookstaging.s3.amazonaws.com/ffmpeg-git-amd64-static.tar.xz

and redeploy. Before redeploying I also restarted the dynos, but I guess that it is not necessary.

This solution is not safe. Do not use it for enterprises and banks.

[sasharevzin]

@kayvaninvemo is it safe to use this bucket? it looks very suspicious to me

[agieche]

@kayvaninvemo is it safe to use this bucket? it looks very suspicious to me

Its a personal bucket by me. Feel free to upload the archive on your own space and point the env variable to your personal url. This makes it more robust. For the time being, you may use my copy, since Im planning to keep it up until things have been sorted out by heroku support.

[diegotorpoco]

@agieche What value should the env variable FFMPEG_DOWNLOAD_URL in heroku client should take? many thanks!

Just add a new env variable in heroku->setting FFMPEG_DOWNLOAD_URL : cookycookstaging.s3.amazonaws.com/ffmpeg-git-amd64-static.tar.xz

and redeploy. Before redeploying I also restarted the dynos, but I guess that it is not necessary.

Hey, thanks! In the mean time I tried to use @dzuelke recommendation of using heroku active storage buildpack (#72)

I have tested in production and it works.

https://github.com/heroku/heroku-buildpack-activestorage-preview.git on heroku buildpacks section

[kayvaninvemo]

@kayvaninvemo is it safe to use this bucket? it looks very suspicious to me

It is not safe, do not use it if it is for an enterprise or a bank. :D

[palinair]

Hi! We have had the same problem in our project. Thanks to those who are sharing their valuable solutions!

BTW, I did some tests to download the file via curl (using options similar to this buildpack) and on a few times it failed for 2 different reasons:

$ curl "https://johnvansickle.com/ffmpeg/builds/ffmpeg-git-amd64-static.tar.xz" -L --fail --retry 5 --retry-max-time 15 -o ./ffmpeg.tar.xz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (35) Unknown SSL protocol error in connection to johnvansickle.com:443

and

$ curl "https://johnvansickle.com/ffmpeg/builds/ffmpeg-git-amd64-static.tar.xz" -L --fail --retry 5 --retry-max-time 15 -o ./ffmpeg.tar.xz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
 69 39.4M   69 27.2M    0     0   520k      0  0:01:17  0:00:53  0:00:24  593k
curl: (18) transfer closed with 12809142 bytes remaining to read

[cjreeve]

Any update on this? Is this for Heroku to fix?

[cjreeve]

As a quick work around for hosting the ffmpeg-git-amd64-static.tar.xz file, I copied the file into my development public folder, started ngrok and set the Heroku config var FFMPEG_DOWNLOAD_URL to the secure ngrok link to the file.

[agieche]

Got a response by heroku. Seems to be something, that the remote site could fix.

It seems that the issue you're encountering is related to a connection error when trying to retrieve the file from the specified URL. The error message "OpenSSL SSL_connect: Connection reset by peer" indicates a connection reset from the remote server (johnvansickle.com) during the SSL handshake.

This type of error is typically caused by issues on the remote server's end, rather than something you can fix from your side. It could be due to temporary server problems, network issues, or misconfigurations on their part.

To troubleshoot, you might want to:

Double-check the URL to ensure it's correct.
Confirm that the remote server is up and running.
Attempt the download from a different network or location to see if the issue persists.
If the problem persists, reaching out to the website's administrator or support might be the best course of action to get more information about the issue and any potential fixes.

[edmorley]

This issue is not related to Heroku.

The server's TLS handling seems broken (or there is otherwise some other issue with the server) - local requests fail too depending on the client used, for example this is what I get using curl locally on macOS:

$ curl -v https://johnvansickle.com/ffmpeg/builds/ffmpeg-git-amd64-static.tar.xz -O
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 107.180.57.212:443...
* Connected to johnvansickle.com (107.180.57.212) port 443 (#0)
* ALPN: offers h2,http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
} [322 bytes data]
*  CAfile: /etc/ssl/cert.pem
*  CApath: none
* Recv failure: Connection reset by peer
* LibreSSL/3.3.6: error:02FFF036:system library:func(4095):Connection reset by peer
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
* Closing connection 0
curl: (35) Recv failure: Connection reset by peer

Ultimately I think this buildpack is not a great fit for production workloads, since:

• it sources the binaries from an arbitrary third party server, which:
  • could result in supply chain attack issues (a compromise of that site, or even just the domain say expiring could mean all secrets in your app are stolen for example),
  • doesn't seem to be very reliable (this isn't the first time downloads from that site have failed)
• it always downloads "latest" ffmpeg, which means your builds or app could fail at any time, if there is a new major release that is not compatible with your app
• it doesn't use the build cache at all, which is fine for apps using a reliable source for assets (such as an S3 bucket), but less so for downloads from third party sites (as proven by builds failing now due to the hard dependency; plus this buildpack presumably costs the site owner a bunch of bandwidth)

I'd recommend that people either:

1. Switch to https://github.com/heroku/heroku-buildpack-activestorage-preview - which is aimed at Ruby Rails apps, but includes ffmpeg so should work fine. And whilst it's not an "officially supported language buildpack" it's still fairly regularly maintained and uses the same security best practices of the official Heroku buildpacks.
2. Use the APT buildpack
3. Download https://johnvansickle.com/ffmpeg/builds/ffmpeg-git-amd64-static.tar.xz yourselves, host somewhere secure of your chosing (your own S3 bucket), then reference using the newly fixed FFMPEG_DOWNLOAD_URL

[keShraa]

Thanks @edmorley for the very detailed explanation 🙏🏻

[ByTE1974]

Looks like it's working again!

[jonathanong]

If you are using this in production, I would definitely store the binary in my own file storage. you don't want to rely on third party services like this. When I was using this repo, I was using it at an early stage start up.

thanks for the help @agieche, let me know if you can help @edmorley. please email me if any issues come up because I don't code nowadays