mirror of
https://github.com/movie-web/providers.git
synced 2025-09-13 17:43:25 +00:00
switched to guider
This commit is contained in:
74
.docs/pages/essentials/customize-providers.mdx
Normal file
74
.docs/pages/essentials/customize-providers.mdx
Normal file
@@ -0,0 +1,74 @@
|
||||
# Customize providers
|
||||
|
||||
You make the provider controls in two ways. Either with `makeProviders()` (the simpler option) or with `buildProviders()` (more elaborate and extensive option).
|
||||
|
||||
## `makeProviders()` (simple)
|
||||
|
||||
To know what to set the configuration to, you can read [How to use on X](./0.usage-on-x.md) for a detailed guide on how to configure your controls.
|
||||
|
||||
```ts
|
||||
const providers = makeProviders({
|
||||
// fetcher, every web request gets called through here
|
||||
fetcher: makeStandardFetcher(fetch),
|
||||
|
||||
// proxied fetcher, if the scraper needs to access a CORS proxy. this fetcher will be called instead
|
||||
// of the normal fetcher. Defaults to the normal fetcher.
|
||||
proxiedFetcher: undefined;
|
||||
|
||||
// target of where the streams will be used
|
||||
target: targets.NATIVE;
|
||||
|
||||
// Set this to true, if the requests will have the same IP as
|
||||
// the device that the stream will be played on.
|
||||
consistentIpForRequests: false;
|
||||
})
|
||||
|
||||
```
|
||||
|
||||
## `buildProviders()` (advanced)
|
||||
|
||||
To know what to set the configuration to, you can read [How to use on X](./0.usage-on-x.md) for a detailed guide on how to configure your controls.
|
||||
|
||||
### Standard setup
|
||||
|
||||
```ts
|
||||
const providers = buildProviders()
|
||||
.setTarget(targets.NATIVE) // target of where the streams will be used
|
||||
.setFetcher(makeStandardFetcher(fetch)) // fetcher, every web request gets called through here
|
||||
.addBuiltinProviders() // add all builtin providers, if this is not called, no providers will be added to the controls
|
||||
.build();
|
||||
```
|
||||
|
||||
### Adding only select few providers
|
||||
|
||||
Not all providers are great quality, so you can make an instance of the controls with only the providers you want.
|
||||
|
||||
```ts
|
||||
const providers = buildProviders()
|
||||
.setTarget(targets.NATIVE) // target of where the streams will be used
|
||||
.setFetcher(makeStandardFetcher(fetch)) // fetcher, every web request gets called through here
|
||||
.addSource('showbox') // only add showbox source
|
||||
.addEmbed('febbox-hls') // add febbox-hls embed, which is returned by showbox
|
||||
.build();
|
||||
```
|
||||
|
||||
|
||||
### Adding your own scrapers to the providers
|
||||
|
||||
If you have your own scraper and still want to use the nice utilities of the provider library or just want to add on to the built-in providers, you can add your own custom source.
|
||||
|
||||
```ts
|
||||
const providers = buildProviders()
|
||||
.setTarget(targets.NATIVE) // target of where the streams will be used
|
||||
.setFetcher(makeStandardFetcher(fetch)) // fetcher, every web request gets called through here
|
||||
.addSource({ // add your own source
|
||||
id: 'my-scraper',
|
||||
name: 'My scraper',
|
||||
rank: 800,
|
||||
flags: [],
|
||||
scrapeMovie(ctx) {
|
||||
throw new Error('Not implemented');
|
||||
}
|
||||
})
|
||||
.build();
|
||||
```
|
74
.docs/pages/essentials/fetchers.mdx
Normal file
74
.docs/pages/essentials/fetchers.mdx
Normal file
@@ -0,0 +1,74 @@
|
||||
# Fetchers
|
||||
|
||||
When creating provider controls, a fetcher will need to be configured.
|
||||
Depending on your environment, this can come with some considerations:
|
||||
|
||||
## Using `fetch()`
|
||||
In most cases, you can use the `fetch()` API. This will work in newer versions of Node.js (18 and above) and on the browser.
|
||||
|
||||
```ts
|
||||
const fetcher = makeStandardFetcher(fetch);
|
||||
```
|
||||
|
||||
If you using older version of Node.js. You can use the npm package `node-fetch` to polyfill fetch:
|
||||
|
||||
```ts
|
||||
import fetch from "node-fetch";
|
||||
|
||||
const fetcher = makeStandardFetcher(fetch);
|
||||
```
|
||||
|
||||
## Using fetchers on the browser
|
||||
When using this library on a browser, you will need a proxy. Browsers restrict when a web request can be made. To bypass those restrictions, you will need a CORS proxy.
|
||||
|
||||
The movie-web team has a proxy pre-made and pre-configured for you to use. For more information, check out [movie-web/simple-proxy](https://github.com/movie-web/simple-proxy). After installing, you can use this proxy like so:
|
||||
|
||||
```ts
|
||||
const fetcher = makeSimpleProxyFetcher("https://your.proxy.workers.dev/", fetch);
|
||||
```
|
||||
|
||||
If you aren't able to use this specific proxy and need to use a different one, you can make your own fetcher in the next section.
|
||||
|
||||
## Making a derived fetcher
|
||||
|
||||
In some rare cases, a custom fetcher is necessary. This can be quite difficult to make from scratch so it's recommended to base it off of an existing fetcher and building your own functionality around it.
|
||||
|
||||
```ts
|
||||
export function makeCustomFetcher(): Fetcher {
|
||||
const fetcher = makeStandardFetcher(f);
|
||||
const customFetcher: Fetcher = (url, ops) => {
|
||||
// Do something with the options and URL here
|
||||
return fetcher(url, ops);
|
||||
};
|
||||
|
||||
return customFetcher;
|
||||
}
|
||||
```
|
||||
|
||||
If you need to make your own fetcher for a proxy, ensure you make it compatible with the following headers: `Set-Cookie`, `Cookie`, `Referer`, `Origin`. Proxied fetchers need to be able to write/read those headers when making a request.
|
||||
|
||||
|
||||
## Making a fetcher from scratch
|
||||
|
||||
In some rare cases, you need to make a fetcher from scratch.
|
||||
This is the list of features it needs:
|
||||
- Send/read every header
|
||||
- Parse JSON, otherwise parse as text
|
||||
- Send JSON, Formdata or normal strings
|
||||
- get final destination URL
|
||||
|
||||
It's not recommended to do this at all. If you have to, you can base your code on the original implementation of `makeStandardFetcher`. Check out the [source code for it here](https://github.com/movie-web/providers/blob/dev/src/fetchers/standardFetch.ts).
|
||||
|
||||
Here is a basic template on how to make your own custom fetcher:
|
||||
|
||||
```ts
|
||||
const myFetcher: Fetcher = (url, ops) => {
|
||||
// Do some fetching
|
||||
return {
|
||||
body: {},
|
||||
finalUrl: '',
|
||||
headers: new Headers(), // should only contain headers from ops.readHeaders
|
||||
statusCode: 200,
|
||||
};
|
||||
}
|
||||
```
|
3
.docs/pages/essentials/index.tsx
Normal file
3
.docs/pages/essentials/index.tsx
Normal file
@@ -0,0 +1,3 @@
|
||||
import { createRedirect } from '@neato/guider/client';
|
||||
|
||||
export default createRedirect({ to: '/essentials/usage-on-x' });
|
14
.docs/pages/essentials/targets.mdx
Normal file
14
.docs/pages/essentials/targets.mdx
Normal file
@@ -0,0 +1,14 @@
|
||||
# Targets
|
||||
|
||||
When creating provider controls, you will immediately be required to choose a target.
|
||||
|
||||
<Warning>
|
||||
A target is the device on which the stream will be played.
|
||||
**Where the scraping is being run has nothing to do with the target**, only where the stream is finally played in the end is significant in choosing a target.
|
||||
</Warning>
|
||||
|
||||
#### Possible targets
|
||||
- **`targets.BROWSER`** Stream will be played in a browser with CORS
|
||||
- **`targets.BROWSER_EXTENSION`** Stream will be played in a browser using the movie-web extension (WIP)
|
||||
- **`targets.NATIVE`** Stream will be played on a native video player
|
||||
- **`targets.ANY`** No restrictions for selecting streams, will just give all of them
|
72
.docs/pages/essentials/usage-on-x.mdx
Normal file
72
.docs/pages/essentials/usage-on-x.mdx
Normal file
@@ -0,0 +1,72 @@
|
||||
# How to use on X
|
||||
|
||||
The library can run in many environments, so it can be tricky to figure out how to set it up.
|
||||
|
||||
Here is a checklist. For more specific environments, keep reading below:
|
||||
- When requests are very restricted (like browser client-side). Configure a proxied fetcher.
|
||||
- When your requests come from the same device on which it will be streamed (not compatible with proxied fetcher). Set `consistentIpForRequests: true`.
|
||||
- To set a target. Consult [Targets](./1.targets.md).
|
||||
|
||||
To make use of the examples below, check out the following pages:
|
||||
- [Quick start](../1.get-started/1.quick-start.md)
|
||||
- [Using streams](../2.essentials/4.using-streams.md)
|
||||
|
||||
## NodeJs server
|
||||
```ts
|
||||
import { makeProviders, makeStandardFetcher, targets } from '@movie-web/providers';
|
||||
|
||||
const providers = makeProviders({
|
||||
fetcher: makeStandardFetcher(fetch),
|
||||
target: chooseYourself, // check out https://movie-web.github.io/providers/essentials/targets
|
||||
})
|
||||
```
|
||||
|
||||
## Browser client-side
|
||||
|
||||
Using the provider package client-side requires a hosted version of simple-proxy.
|
||||
Read more [about proxy fetchers](/essentials/fetchers#using-fetchers-on-the-browser).
|
||||
|
||||
```ts
|
||||
import { makeProviders, makeStandardFetcher, targets } from '@movie-web/providers';
|
||||
|
||||
const proxyUrl = "https://your.proxy.workers.dev/";
|
||||
|
||||
const providers = makeProviders({
|
||||
fetcher: makeStandardFetcher(fetch),
|
||||
proxiedFetcher: makeSimpleProxyFetcher(proxyUrl, fetch),
|
||||
target: target.BROWSER,
|
||||
})
|
||||
```
|
||||
|
||||
## React native
|
||||
To use the library in a react native app, you would also need a couple of polyfills to polyfill crypto and base64.
|
||||
|
||||
<Steps>
|
||||
<Steps.Step>
|
||||
### First install the polyfills:
|
||||
```sh npm2yarn
|
||||
npm install @react-native-anywhere/polyfill-base64 react-native-quick-crypto
|
||||
```
|
||||
</Steps.Step>
|
||||
<Steps.Step>
|
||||
### Add the polyfills to your app:
|
||||
```ts
|
||||
// Import in your entry file
|
||||
import '@react-native-anywhere/polyfill-base64';
|
||||
```
|
||||
|
||||
And follow the [react-native-quick-crypto documentation](https://github.com/margelo/react-native-quick-crypto) to set up the crypto polyfill.
|
||||
</Steps.Step>
|
||||
<Steps.Step>
|
||||
### Then you can use the library like this:
|
||||
```ts
|
||||
import { makeProviders, makeStandardFetcher, targets } from '@movie-web/providers';
|
||||
|
||||
const providers = makeProviders({
|
||||
fetcher: makeStandardFetcher(fetch),
|
||||
target: target.NATIVE,
|
||||
consistentIpForRequests: true,
|
||||
})
|
||||
```
|
||||
</Steps.Step>
|
||||
</Steps>
|
84
.docs/pages/essentials/using-streams.mdx
Normal file
84
.docs/pages/essentials/using-streams.mdx
Normal file
@@ -0,0 +1,84 @@
|
||||
# Using streams
|
||||
|
||||
Streams can sometimes be quite picky on how they can be used. So here is a guide on how to use them.
|
||||
|
||||
## Essentials
|
||||
|
||||
All streams have the same common parameters :
|
||||
- `Stream.type` : The type of stream. Either `hls` or `file`
|
||||
- `Stream.id` : The id of this stream, unique per scraper output.
|
||||
- `Stream.flags` : A list of flags that apply to this stream. Most people won't need to use it.
|
||||
- `Stream.captions` : A list of captions/subtitles for this stream.
|
||||
- `Stream.headers` : Either undefined or a key value object of headers you must set to use the stream.
|
||||
- `Stream.preferredHeaders` : Either undefined or a key value object of headers you may want to set if you want optimal playback - but not required.
|
||||
|
||||
Now let's delve deeper into how to watch these streams!
|
||||
|
||||
## Streams with type `hls`
|
||||
|
||||
HLS streams can be tough to watch. They're not normal files you can just use.
|
||||
These streams have an extra property `Stream.playlist` which contains the m3u8 playlist.
|
||||
|
||||
Here is a code sample of how to use HLS streams in web context using hls.js
|
||||
|
||||
```html
|
||||
<script src="https ://cdn.jsdelivr.net/npm/hls.js@1"></script>
|
||||
|
||||
<video id="video"></video>
|
||||
<script>
|
||||
const stream = null; // add your stream here
|
||||
|
||||
if (Hls.isSupported()) {
|
||||
var video = document.getElementById('video');
|
||||
var hls = new Hls();
|
||||
hls.loadSource(stream.playlist);
|
||||
hls.attachMedia(video);
|
||||
}
|
||||
</script>
|
||||
```
|
||||
|
||||
## Streams with type `file`
|
||||
|
||||
File streams are quite easy to use, they just return a new property : `Stream.qualities`.
|
||||
This property is a map of quality and a stream file. So if you want to get 1080p quality you do `stream["1080"]` to get your stream file. It will return undefined if that quality is absent.
|
||||
|
||||
The possibly qualities are : `unknown`, `360`, `480`, `720`, `1080`, `4k`.
|
||||
File based streams are always guaranteed to have one quality.
|
||||
|
||||
Once you get a streamfile, you have the following parameters :
|
||||
- `StreamFile.type` : Right now it can only be `mp4`.
|
||||
- `StreamFile.url` : The URL linking to the video file.
|
||||
|
||||
Here is a code sample of how to watch a file based stream in a browser :
|
||||
|
||||
```html
|
||||
<video id="video"></video>
|
||||
<script>
|
||||
const stream = null; // add your stream here
|
||||
const video = document.getElementById('video');
|
||||
|
||||
const qualityEntries = Object.keys(stream.qualities);
|
||||
const firstQuality = qualityEntries[0];
|
||||
video.src = firstQuality.url;
|
||||
</script>
|
||||
```
|
||||
|
||||
## Streams with headers
|
||||
|
||||
Streams have both a `Stream.headers` and a `Stream.preferredHeaders`.
|
||||
The difference between the two is that `Stream.headers` **must** be set in order for the stream to work. While the other is optional, and enhances the quality or performance.
|
||||
|
||||
If your target is set to `BROWSER`, headers will never be required, as it's not possible to do.
|
||||
|
||||
## Using captions/subtitles
|
||||
|
||||
All streams have a list of captions at `Stream.captions`. The structure looks like this :
|
||||
```ts
|
||||
type Caption = {
|
||||
type : CaptionType; // Language type, either "srt" or "vtt"
|
||||
id : string; // Unique per stream
|
||||
url : string; // The URL pointing to the subtitle file
|
||||
hasCorsRestrictions : boolean; // If true, you will need to proxy it if you're running in a browser
|
||||
language : string; // Language code of the caption
|
||||
};
|
||||
```
|
Reference in New Issue
Block a user