gallery-dl

Active development has moved to Codeberg
(see [Announcement] Moving to Codeberg for details)

https://codeberg.org/mikf/gallery-dl/releases/tag/v1.32.1

Extractors

• Additions
  • [aryion] add extractor for watched users (cb#71)
  • [framedsc] add support (pr#58 gh#9192)
• Removals
  • [comick] remove module
  • [weebdex] remove module
• aryion
  • rename watch extractor to messagepage
• blogger
  • fix video extraction (gh#9487)
• deviantart
  • improve CAPTCHA detection during login (gh#9351 cb#9)
• discord
  • cache server API results (cb#57)
  • process server channels as children (cb#57)
• exhentai
  • implement "source": "torrent" (gh#2996 gh#9470)
• facebook
  • handle negative fallback-retries values
  • increase loop detection threshold (gh#9495)
• furaffinity
  • fix download URL extraction (gh#9494)
• imxto
  • extract date metadata (gh#9450)
• instagram
  • add pinned option (gh#9492)
  • restore "posts are private" warning (gh#9291)
  • support audio from clips_metadata (gh#9477)
  • support downloading audio from dash_manifest data
  • unset wd cookie (gh#8300 cb#73)
• nitter
  • add fallback-retries option (cb#59)
  • use request_location() for user ID lookups
• philomena
  • add comments option (gh#9478)
• pixiv
  • fix & update search extractor (gh#9469)
• reddit
  • add pinned option (gh#9004)
• scrolller
  • fix user GraphQL query
• telegraph
  • extract full article html & links metadata (gh#9468)
• twibooru
  • add comments option (gh#9478)
• twitter
  • fix AttributeError when extended user metadata fetch fails (pr#76)
• vipergirls
  • support proxy domains (cb#43)
• warosu
  • fix extraction (gh#9485)
• weibo
  • fix KeyError: 'title' (cb#47)

Documentation

• replace more GitHub links

Output

• add units option (cb#56)

Scripts

• release
  • include requirements directory
• rm
  • delete extractor/utils files

Update

• pull from Codeberg for versions >= 1.32.0 (gh#9473 gh#9496)

Utilities

• job
  • run filter before date & tags predicates (gh#9492)
• text
  • implement iextr()
• util
  • implement format_bytes_binary()

Active development has moved to Codeberg
(see [Announcement] Moving to Codeberg for details)

https://codeberg.org/mikf/gallery-dl/releases/tag/v1.32.0

Core

• implement basic server/IPC queue (pr#9338)
• implement date-before/-after options (gh#1167 gh#1389 gh#2308)
• implement tags-blacklist/-whitelist options (gh#2446 gh#3755 gh#4240 gh#4248 gh#4270 gh#4272 gh#4387 gh#5208 gh#6267 gh#6845 gh#7324 gh#7910)
• implement follow option (gh#8752)
• implement path-restricted archive paths (gh#3359 gh#3361 gh#4100 gh#8027)
• implement linear/exponential backoff for sleep-429
• implement lazy-loading of larger extractor resources like GraphQL queries and HTML generation functionality
• refactor internal caching mechanisms
• rename image-* & chapter-* options to file-* & child-*
• enable metadata-url/-path/-parent/-extractor by default
• remove date-format option - use ISO 8601 format instead
• report errors when using removed/deprecated options (gh#9163)
• fix preserving (private) metadata in cached values (gh#9375)

Extractors

common

• add common headers to request_json() requests
• add reference to exception module to Extractor class
• fix "browser": "…/VERSION" parsing (gh#9401)
• improve __NEXT_DATA__ extraction (gh#9420 gh#9435 gh#9447 cb#23 cb#30)
• include duration in wait() output
• exclude alt URLs from include=all

8muses

• use dynamic directories by default

allporncomic

• add support (gh#1961)
• extract title & media metadata (gh#1961)

aryion

• improve error message on login failure (gh#9429)
• use text.filename_from_contentdisposition

bellazon

• ignore /main/style_emoticons/ URLs (gh#9140)

bluesky

• support submitting sign-in code during login (gh#9367)

bunkr

• add direct-link extractor (cb#27)

chevereto

• combine image & video extractors into a file extractor
• improve password requirement detection
• support putmega.com (gh#2045 gh#9361)

cien

• support downloading article cover images

civitai

• support civitai.red (gh#9421 gh#9422 gh#9426 gh#9459 cb#29 cb#32)
  (api-key or cookies are now required to access NSFW content)

comicartfans

• add support (gh#4524)
• implement child-range skip

coomer

• improve default file order (gh#9097)

cosmos

• add support (gh#9358)
• fix KeyError: 'multipleMedia' (gh#9358)
• update default image format
• support specifying a user ID with id:...

cosplayrule34

• add support (pr#5522 gh#9412 pr#9396)

cyberdrop

• support /e/ embed URLs (gh#9301)

danbooru

• implement tags-blacklist import (gh#5208)

deviantart

• emit log messages when requesting download URL (gh#9217)
• export journal templates & tiptap functions
• fix login with username & password (gh#9351)
• fix possible KeyError: 'extension' (gh#9339)
• fix TypeError when auto-unwatch is enabled (gh#9182)
• fix "refresh-token": "cache" (gh#9217 gh#9268)
• implement code_challenge flow for oauth:deviantart (pr#14)

discord

• [search] implement date-before skip

e621

• implement tags-blacklist import (gh#5208)
• update tags format

exhentai

• enable tags categories by default

facebook

• fix TypeError for users without avatar page (gh#9347)
• change loop default to true (cb#26)
• disable jump detection for multi-photo posts (gh#9286)
• download only 1 file for single-photo posts (gh#9286)
• support /permalink.php post URLs (gh#9352)

flickr

• fix extracting public API key (gh#9449)
• use non-download URLs by default (gh#8297)

foolfuuka

• improve media link resolution (pr#9047)

gelbooru_v01

• support scatbooru.co.uk (gh#9239 gh#9300)

gelbooru_v02

• fix infinite pagination loop when using --range (gh#9176)

gofile

• update WT generation salt (gh#9183)
• enable recursive by default

harvardlawnuremberg

• add document extractor (pr#18)

imagepond

• support /i/ URLs & update root domain
• update extractors

imageshack

• add support (gh#9282)

instagram

• add audio option (gh#9297)
• extract audio metadata (gh#9297)
• handle missing music_info and skip invalid audio URLs (gh#9335)
• add photos extractor (pr#25)
• add static-videos option (gh#9103)
• add user-cache option (gh#8978 gh#9025)
• add user-strategy option (gh#8978 gh#9025)
• expose story item audience metadata (gh#9320)
• extend previews option (gh#9297)
• extend user-strategy option (gh#9039 gh#9069)
• fix user caching behavior (gh#9025)

joyreactor

• rewrite (gh#6642 gh#7104 gh#7531)
• ignore tag info section
• stop pagination when next page link is not present

koofr

• fix names with special characters (cb#31)

mangadex

• extract user metadata (pr#15)

myhentaigallery

• support URLs starting with /a/ (cb#13)

newgrounds

• fix login with username & password (gh#9308)

nhentai

• update to API v2 (cb#1)

nsfwalbum

• add image extractor (gh#9372)
• implement file-range skip (gh#9372)

nudostar

• fix file URLs of models with more than 1000 images (gh#9332)

oauth

• fix AttributeError: 'update' (gh#9230)
• fix cache attribute overwriting function
• implement code_challenge flow for deviantart (pr#14)

patreon

• fix extracting content images (gh#9292)
• restore content metadata (gh#9171 gh#9208 gh#9185)
• use text.filename_from_contentdisposition (gh#9322)
• [user] implement date-before skip

pholder

• fix solving challenges (gh#9264)

pixeldrain

• enable recursive by default

pixiv

• fix KeyError: 'total_comments' (cb#16)
• rename follow to followed subcategory
• [bookmark] implement cursor support (gh#9289)

plurk

• refactgor (gh#1111 gh#1683 gh#4360 gh#4425 gh#5477 pr#5)
• add directory_fmt & filename_fmt
• add plurk_id_base36 in metadata
• add option external & replurk options
• fix directlink error
• parse dates

poipiku

• implement login with username & password (gh#9284)

postype

• add support (pr#17)

reddit

• add user-agent-oauth option (gh#8641 gh#8838)
• support www.old.reddit.com URLs (gh#9137)
• use OAuth API when client_id is given (gh#8838)
• [user] improve error message for suspended accounts (gh#9177)

redgifs

• implement post-range skip (gh#9274)
• improve API related error messages (gh#9274)
• increase number of results per page to 100 (gh#9274)
• [user] forward URL parameters (gh#9274)

sankaku

• implement tags-blacklist import (gh#5208)

schalenetwork

• enable tags categories by default

scrolller

• add user extractor (gh#8961)

sizebooru

• fix id extraction (gh#9425)

snapchat

• add support (gh#7126 pr#24)

szurubooru

• submit consent form when necessary
• use util.HTTPBasicAuth()

tcbscans

• support tcbonepiecechapters.com (gh#9380 pr#3)

tsumino

• remove module

tumblr

• do not ignore reblogs option for single posts
• filter duplicate inline images (gh#9418)
• fix "reblogs": "same-blog" (gh#9360)
• improve inline file regex (gh#6505)
• retry without OAuth when blocked by other user (gh#8803)
• support date-before skip

twitter

• ensure self.api is always defined (gh#9271 gh#9273 cb#9295)
• extend article handling (gh#8995)
• extract friends_mutual user metadata (gh#9349)
• prevent crash when processing empty pinned Tweets (#9231)
• fix KeyError: 'id_str' when downloading card videos (gh#9294)
• fix possible KeyError: 'entities' in _transform_user()
• implement metadata-user option (gh#8984)
• implement "search-pagination": "date"
• implement showreplies option (gh#7092 gh#9389)
• rename replies subcategory to with-replies (gh#9162 gh#8107)
• rewrite article image URLs (gh#8995)
• use _tweetid_to_datetime method

urlgalleries

• update extractor (gh#9090)

vipergirls

• support viper.to URLs (gh#9417)

webtoons

• extract date metadata from comic episode listings (pr#6)

weibo

• add likes option (gh#9366)
• update Sina Visitor System bypass (gh#9279)
• visit homepage for guest cookies when API blocked (gh#9279)

xenforo

• decode /redirect/ URLs (gh#9042 gh#9364)
• extract thread[path] metadata (gh#9369)
• fix binascii.Error: Incorrect padding
• improve detection of auth requirement (gh#9216)
• support thefappeningblog.com/forum (cb#12)

xfolio

• implement fullsize option
• support .webp preview image downloads

ytdl

• propagate ControlException and handle ignoreerrors (pr#9229)

zerochan

• fix /fav/ tag searches (gh#9387)

Downloaders

http

• check download flag before connecting to server (gh#9109)

Post Processors

• add child & child-after events

exec

• add output option
• implement success & error options

metadata

• add empty option
• improve _write_tags() code

ugoira

• add more mkvmerge options (gh#8803)
• disable mkvmerge-mtime by default (gh#9222)
• disable mkvmerge output, add mkvmerge-output option

Actions

• implement keyword action

flag

• implement skip (gh#8960)
• implement pause, toggle, and clear
• add ability to clear/reset all flags

Cache

• remove decorator classes
• support :memory: database path
• update database file access

Configuration

• support setting default file type via GDL_CONFIG_TYPE environment variable and --config-type command-line option
• improve --config-status
• fix JSONDecodeError when using non-JSON config types (gh#9063)

Docker

• harden builds
  • enable hash-checking mode
  • disallow source distributions
  • https://pip.pypa.io/en/stable/topics/secure-installs/
• install gallery-dl from Codeberg repository

Documentation

• add pull_request_template.md

configuration

• document postprocessor /MODE and @EVENT (gh#8980)
• document postprocessor type (gh#8803)
• improve output.colors & output.ansi description (gh#9155)
• update extractor.*.archive
• use anchors for alternative/deprecated names

README

• fix broken repo links (pr#2)
• include link to options in opening paragraph (gh#8803)
• update GitHub links (cb#38)
• update links to stable executable releases

Options

• add --blacklist & --whitelist command-line options
• add --config-type & --config-json command-line options
• add --restrict-filenames & --windows-filenames command-line options
• add geo-bypass / --xff option
• add sleep-retries option
• use "ugoira": "original" by default
• use yuv420p output pixel format for ugoira presets
• implement various cache command-line options
  • --cache-clear
  • --cache-file
  • --cache-show
  • --cache-status
  • --cache-vacuum

Path

• implement generate_path()
• include "html": "htm" in default extension-map
• include | replacement in windows+ preset

Scripts

• implement requirements.py
• [pull-request] support Codeberg PR URLs
• [release] update issue link replacements
• [run_tests] randomize test order

Tests

• fix failure due to test order

job

• add tests for DataJob resolve
• test parent-metadata / _extractor handling

results

• fix testing external extractors
• implement testing exception messages

Utilities

text

• implement extract_urls() helper
• implement filename_from_contentdisposition (gh#4022)
• implement parse_hex_escapes

util

• add type argument to HTTPBasicAuth
• implement build_duration_func_ex()
• implement predicate_date() & predicate_tags()
• implement complex tags-blacklist filter rules
• improve range predicate stop by using flags
• remove support for paths as lists

Miscellaneous

• add init option
• [exception] implement StopDownload
• [formatter] add q & Q conversions - URL-encode & -decode values
• [workflows:executables] use Python 3.14

Extractors

Additions

• [fantia] add extractor for supported fanclubs (#9236 #9237)
• [foriio] add support (#2293)
• [iwara] support iwara.ai (#9241)
• [rule34world] add support (#9227)
• [wikimedia] support mgewiki.moe (#9258)
• [xenforo] support blacktowhite.net (#9216)

Fixes

• [bellazon] handle /main/index.php attachments (#9140)
• [bilibili] fix & improve livephoto filenames (#9210)
• [deviantart] disable original downloads by default (#9211 #9213 #9217)
• [deviantart] apply original to all downloadable file types (#9217)
• [exhentai] prevent ValueError for incomplete MPV data (#9259)
• [gofile] update Website Token salt
• [myportfolio] disallow cdn.myportfolio.com URLs
• [patreon] restrict legacy User-Agent use (#9249)
• [pholder] fix Could not locate window.data JSON error (#9264)
• [reddit] fix results for /user/…/comments/ URLs (#9212)
• [reddit] fix previews for comment submissions (#9206)
• [twitter] fix ondemand.s.a.js extraction (#9260 #9262 #9265 #9267 #9270)
• [twitter] prevent crash when processing empty pinned Tweets (#9231)

Improvements

• [bilibili] extract user_id metadata (#9210)
• [deviantart] remove blur filters (#9217)
• [twitter] implement previews option (#9232)
• [wikimedia] handle index.php paths (#9258)
• [xenforo] add quoted option (#9207)
• [xenforo] add attachments & embeds options
• [xenforo] improve auth requirement detection (#9216)
• [yiffverse] recognize system & meta tag types

Post Processors

• [mtime] allow using mode to select date key

Extractors

Fixes

• [twitter] fix AttributeError: '_update_variables_search_maxid' (#9203)

Extractors

Additions

• [fanbox] add tag extractor (#9128)
• [filester] add support (#9196)
• [mixdrop] add file extractor (#8982 #9130)
• [readcomiconline] add tag extractor (#9190)

Fixes

• [bellazon] fix content of posts with multiple quotes (#9140)
• [fansly] fix KeyError: 'data' (#9149)
• [gofile] fix extractor (#9183)
• [kemono:discord] fix archive data overwriting general data (#9184 #9195)
• [luscious] fix empty file URLs (#9172 #9188)
• [paheal] fix tag searches with only 1 result (#9186)
• [patreon] update default User-Agent headers (#9165)
• [reddit:user] disable only for upvoted & downvoted results (#9124)
• [simplyhentai] rewrite (#9158)
• [tiktok] use more lenient regex for secondary user IDs (#9094)
• [twitter] use max_id as default search-pagination (#9193 #9194)
• [weebdex] fix KeyError: 'chapter' for oneshots (#9123)
• [whyp] fix downloading private content (#9133)

Improvements

• [e621] support shared post URLs (#9168)
• [sankaku] include parent post in parent:… searches (#9102)
• [smugmug] improve error for deleted files (#9140)
• [tumblr] support media from answer'/'asks posts (#4509)
• [vipr] detect missing images (#8415)
• [xenforo] support imgur media embeds (#9127)

Metadata

• [facebook:photo] improve metadata availability (#9147)
• [pixiv] extract hash metadata when available
• [postimg] fix filename metadata (#9119)

Downloaders

• [http] release connections on 416 Range Not Satisfiable responses

Post Processors

• [actions] add actions postprocessor
• [hash] allow using mode to select hashes
• [ugoira] add more mkvmerge options (#8803)

Miscellaneous

• [path] add windows+ as a special path-restrict value