Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

This is no longer working due to Just watch API changes #66

Open
voodooenglishman opened this issue Oct 11, 2023 · 21 comments
Open

This is no longer working due to Just watch API changes #66

voodooenglishman opened this issue Oct 11, 2023 · 21 comments

Comments

@voodooenglishman
Copy link

I believe this no longer works could be wrong as I can't find an older version of the justwatch API documentation but I believe its changed since the last update to this

Tried running one of my usual scripts and was given errors down to this API in my code

Was getting 404 errors

https://apis.justwatch.com/docs/api/

Looking at the current API and the code from this it seems the URLs are no longer pointing correctly

Example being the new base URL which is "https://apis.justwatch.com/contentpartner/v2/content" which is different from the one given within this API

Thank you

@NRaabjerg
Copy link

I am experiencing the same issue. I believe JustWatch have closed public API access. I will try to look further into it, but hopefully someone will be able to get around it at some point.

@dseomn
Copy link

dseomn commented Oct 11, 2023

I'm getting 404s trying to access content and search pages, but https://apis.justwatch.com/content/locales/state and https://apis.justwatch.com/content/providers/locale/en_US are still working for me. Over the past few days I also noticed missing seasons fields in some TV show pages, which all got fixed eventually. Maybe JustWatch is just doing some sort of migration and it's not going smoothly for this API?

But if the current API is going away, lufinkey/node-justwatch-api#17 mentions a graphql API that might work instead?

@byte4geek
Copy link

Same for me

@anemochore
Copy link

Same here. I searched and found reddit post and then I was able to get it work again. (changed whole payload)

dseomn added a commit to dseomn/rock-paper-sand that referenced this issue Oct 13, 2023
It looks like JustWatch removed their old API, and the replacement with
GraphQL works pretty differently in some ways. See
dawoudt/JustWatchAPI#66 for some more details.

Before migrating to the new API, this commit removes features that I
don't think will need to be migrated. The new API seems to use separate
country and language fields instead of the joined locale field, so I
removed the code that lists locales. (The country and language codes
look standard.) The new API also looks like it can easily work with
normal URLs from their website instead of mostly-opaque identifiers, so
I removed the code to search for IDs.
@DiegoFleitas
Copy link

Idem, GraphQL might be the only viable option now

@dseomn
Copy link

dseomn commented Oct 15, 2023

In case it helps anyone, here's some code I got working using the GraphQL API. For what it's worth, the new API feels significantly more powerful than the old API in some ways, though it also wasn't a trivial migration to the new API.

https://github.com/dseomn/rock-paper-sand/blob/70bf3542f65b6a3ca92f0784e64bcc29476a393d/rock_paper_sand/justwatch.py#L64

@anemochore
Copy link

anemochore commented Oct 19, 2023

Well, I suspect if my TamperMonkey script will help, but this is my code. (Somehow reverse engineered with help of reddit post previously mentioned) My scenario is searching by title and then getting original title, year, type, imdb rating, etc. https://github.com/anemochore/imdbOnWatcha/blob/master/utils.js#L80

@dseomn
Copy link

dseomn commented Oct 19, 2023

Edit: sorry for the noise, this was an issue with my code, not with the API. I was caching all "successful" HTTP requests, including ones that return JSON with an errors key. So it looked like the API kept returning the error below, but it actually just returned it once and my code cached that.

I just started getting errors like the below from the graphql API. Note that the HTTP response code was 200 (or some other non-error code), and the mention of 429 is in the response body. Is anybody else seeing something like this with the new API?

{'errors': [{'message': 'rpc error: code = Unknown desc = getting offers: GetTitleOffers: reading from source: error response received from Elasticsearch: 429 Too Many Requests', 'path': ['urlV2', 'node', 'seasons', 0, 'episodes', 5, 'offers'], 'extensions': {'code': 'INTERNAL_ERROR'}}, {'message': 'rpc error: code = Unknown desc = getting offers: GetTitleOffers: reading from source: error response received from Elasticsearch: 429 Too Many Requests', 'path': ['urlV2', 'node', 'seasons', 1, 'episodes', 9, 'offers'], 'extensions': {'code': 'INTERNAL_ERROR'}}], 'data': None}

@anemochore
Copy link

@dseomn Error message says 'Too Many Requests.' Maybe you are requesting too many. In my case, no such errors occured.

@dseomn
Copy link

dseomn commented Oct 23, 2023

See the "Edit: ..." text I added to that comment. It was just a transient issue, probably from too many requests to their backend, not too many requests from me. The fix was easy: dseomn/rock-paper-sand@692e63c

@DiegoFleitas
Copy link

DiegoFleitas commented Oct 26, 2023

Thanks for sharing that demo, @dseomn! By the way, for anyone else tinkering with this, I think I encountered a quirk regarding the newline characters used in GraphQL queries: there's a different behavior when using \r\n versus \n. So, if you come across any strange behavior, checking the newline characters might be a good place to start. I believe the API may have cached an error response for a seemingly correct request. The errors ceased when I switched to using \r\n as the newline character in my queries.

@DanielSouzaBertoldi
Copy link

After reading the code of both projects listed here and the reddit post, I'm still not sure on how to fetch info of a media by passing along only its ID + objectType. Checking out the /graphql requests in the JustWatch site I can only find operations for GetTitlePopularityRank and GetSimilarTitles 🤔 has anyone figured out what's the query for this?

@dseomn
Copy link

dseomn commented Oct 27, 2023

https://github.com/dseomn/rock-paper-sand/blob/9a26853a0c7a5c68a536b30e69e2cc16c5e4f572/rock_paper_sand/justwatch.py#L233 looks up details by node ID, which is basically just object type + object ID. E.g., object type "SHOW" and object ID "42" is node ID "ts42". I think movies are "tm...", seasons are "tss..." and episodes are "tse...", but I didn't double check that.

@DanielSouzaBertoldi
Copy link

DanielSouzaBertoldi commented Oct 27, 2023

@dseomn nice, thanks for the quick reply! Instead of just asking for the offers data I also added the following to the Movie fragment just to test it out:

fragment Movie on Movie {
    __typename
    id
    content(country: $country, language: $language) {
        fullPath
        title
        originalReleaseYear
        originalTitle
        scoring {
        imdbScore
        imdbVotes
        __typename
        }
        externalIds {
        imdbId
        __typename
        }
        __typename
    }
    offers(country: $country, platform: WEB) {
        monetizationType
        availableToTime
        availableFromTime
        package {
            clearName
            technicalName
        }
    }
}

And it worked, nice. The worst part is figuring out what field name to use for a given data.

@naxels
Copy link

naxels commented Nov 21, 2023

For people catching up, the GraphQL API endpoint is: https://apis.justwatch.com/graphql

@bagajohny
Copy link

Can I use this to export my watchlist? if yes how do I do it?

@DiegoFleitas
Copy link

DiegoFleitas commented Dec 11, 2023

Can I use this to export my watchlist? if yes how do I do it?

It's likely you can do it using the graphql API directly but I think you'll have to invest your own time to get somewhere, I can give you a head start for what's worth

query

query GetTitleListV2($country: Country!, $titleListFilter: TitleFilter, $titleListSortBy: TitleListSortingV2! = LAST_ADDED, $titleListType: TitleListTypeV2!, $titleListAfterCursor: String, $watchNowFilter: WatchNowOfferFilter!, $first: Int! = 10, $language: Language!, $sortRandomSeed: Int! = 0, $profile: PosterProfile, $backdropProfile: BackdropProfile, $format: ImageFormat, $platform: Platform! = WEB, $includeOffers: Boolean = false) {
  titleListV2(
    after: $titleListAfterCursor
    country: $country
    filter: $titleListFilter
    sortBy: $titleListSortBy
    first: $first
    titleListType: $titleListType
    sortRandomSeed: $sortRandomSeed
  ) {
    totalCount
    pageInfo {
      startCursor
      endCursor
      hasPreviousPage
      hasNextPage
      __typename
    }
    edges {
      ...WatchlistTitleGraphql
      __typename
    }
    __typename
  }
}

fragment WatchlistTitleGraphql on TitleListEdgeV2 {
  cursor
  node {
    id
    objectId
    objectType
    offerCount(country: $country, platform: $platform)
    offers(country: $country, platform: $platform) @include(if: $includeOffers) {
      id
      presentationType
      monetizationType
      retailPrice(language: $language)
      type
      package {
        id
        packageId
        clearName
        __typename
      }
      standardWebURL
      elementCount
      deeplinkRoku: deeplinkURL(platform: ROKU_OS)
      __typename
    }
    content(country: $country, language: $language) {
      title
      fullPath
      originalReleaseYear
      shortDescription
      scoring {
        imdbScore
        imdbVotes
        tmdbScore
        tmdbPopularity
        __typename
      }
      posterUrl(profile: $profile, format: $format)
      backdrops(profile: $backdropProfile, format: $format) {
        backdropUrl
        __typename
      }
      upcomingReleases(releaseTypes: [DIGITAL]) {
        releaseDate
        __typename
      }
      isReleased
      __typename
    }
    likelistEntry {
      createdAt
      __typename
    }
    dislikelistEntry {
      createdAt
      __typename
    }
    watchlistEntryV2 {
      createdAt
      __typename
    }
    customlistEntries {
      createdAt
      __typename
    }
    watchNowOffer(country: $country, platform: $platform, filter: $watchNowFilter) {
      id
      standardWebURL
      package {
        id
        packageId
        clearName
        __typename
      }
      retailPrice(language: $language)
      retailPriceValue
      currency
      lastChangeRetailPriceValue
      presentationType
      monetizationType
      availableTo
      __typename
    }
    ... on Movie {
      seenlistEntry {
        createdAt
        __typename
      }
      __typename
    }
    ... on Show {
      tvShowTrackingEntry {
        createdAt
        __typename
      }
      seenState(country: $country) {
        seenEpisodeCount
        releasedEpisodeCount
        progress
        caughtUp
        lastSeenEpisodeNumber
        lastSeenSeasonNumber
        __typename
      }
      __typename
    }
    __typename
  }
  __typename
}

vars

{
  "titleListSortBy": "LAST_ADDED",
  "first": 20,
  "sortRandomSeed": 0,
  "platform": "WEB",
  "includeOffers": false,
  "titleListFilter": {
    "ageCertifications": [],
    "excludeGenres": [],
    "excludeProductionCountries": [],
    "objectTypes": [
      "SHOW"
    ],
    "productionCountries": [],
    "genres": [],
    "packages": [],
    "excludeIrrelevantTitles": false,
    "presentationTypes": [],
    "monetizationTypes": []
  },
  "watchNowFilter": {
    "packages": [],
    "monetizationTypes": []
  },
  "language": "es",
  "country": "ES",
  "titleListType": "WATCHLIST",
  "titleListAfterCursor": ""
}

Btw for this kind of query it's likely you'll require to put your own Bearer token along with the headers which I don't think most of queries we've been using need.
Also note objectType can be used to get SHOW / MOVIE on the response & the vars on mine are set to spanish

@puneetverma24
Copy link

puneetverma24 commented Dec 13, 2023

Please share the complete schema information.

@bagajohny
Copy link

@DiegoFleitas
I am trying the method given in the following github repo but I am getting some error. Can you help me understand it?

ijanos/justwatchdumper#1

This is the link to the issue I created. The method given in this repo seems easier to implement for me. So I will be grateful if you can help me.

@DiegoFleitas
Copy link

Please share the complete schema information.

I don't have access to it & I don't think anyone else in this thread does since JustWatch has introspection disabled for their graphql schema, AFAIK everyone just inspected the requests being sent from JustWatch website 🤷‍♂️

@DiegoFleitas I am trying the method given in the following github repo but I am getting some error. Can you help me understand it?

ijanos/justwatchdumper#1

This is the link to the issue I created. The method given in this repo seems easier to implement for me. So I will be grateful if you can help me.

Sorry, you are on your own with that. Plus that discussion is no longer relevant to this repo. You are welcome to ask here and if you explain what you are trying to do with enough details I might be able to help or provide a simpler solution

@BenutzerEinsZweiDrei
Copy link

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests