Autogenerated API

A program to quickly extract links from a url

fast_link_extractor.link_extractor(base_url: str = None, search_subs: bool = None, regex: str = None, ipython: bool = None, no_warning: bool = None, *args, **kwargs)[source]

Extract links from base_url.

to get output in jupyter you need to await the result first

>>> links = await link_extractor(*args)

Parameters:

base_url (str) – URL you want to search
seach_subs (bool) – True is want to search sub-directories (default is True)
regex (str) – filter links based on a regular expression (default is ‘.’)
ipython (bool) – whether you are using ipython or not (default is False)
no_warning (bool) – toggles on/off the await warning message (default is False, only applies to ipython=True)

Returns:

list of files

Return type:

list

Example

>>> url = 'https://www.ncei.noaa.gov/data/sea-surface-temperature-optimum-interpolation/v2.1/access/avhrr/'
>>> links = await link_extractor(url, search_subs=True, regex='.nc$', ipython=True)