Guides

How to start with transco­ding aka fai­rly sim­ple long ge­ne­ral transco­ding gu­ide

In­tro­duction

So, again a transco­ding (en­co­ding – si­mi­liar terms) gu­ide, this time ge­ne­ral and in En­g­lish. This one should serve as an equi­va­lent for the Czech one i wrote some time ago (al­though much newer and up‐to‐date).

Note that this gu­ide is for Win­dows (XP and la­ter), since you efecti­vely can’t transcode on Li­nux and other sys­tems (in other words – Avi­synth is for Win).
Also note that most of this gu­ide fo­cus­ses on vi­deo transco­ding. Au­dio usu­ally can be de­mu­xed from source and mu­xed to out­put pretty easily.

I should stress out that I fo­cus on transco­ding of anime (Ja­pa­nese ani­ma­ted movies/series) and so does this gu­ide. There are sli­ght di­f­fe­ren­cies in settings requi­red to meet sa­tis­fy­ing qua­lity with live‐action stuff. However, the ba­sic prin­ci­ples de­scri­bed here ap­ply for any en­co­ding.

Images used here are only ilu­strati­o­nal and should give ge­ne­ral idea how the things should look like. For code exam­ples re­fer to co­des I have in the text.

Tools Requi­red

x264.exe – en­co­der (cho­o­sing ver­si­ons: cho­ose 32bit ver­sion since Avi­synth is 32bit; 8bit/10bit ver­sion ac­cor­ding to how wide com­pa­ti­bi­lity you need – these days most PCs will play 10bit en­co­ding, but some de­vi­ces mi­ght not)
Avi­synth – fra­me­ser­ver
no­te­pad or other sim­ple txt edi­tor (not MS Word or so­me­thing like that!)
Vir­tu­al­Dub – for chec­king the script be­fore actual transco­ding

If you want to transcode DVDs (already decryp­ted), then ad­di­ti­o­nally these:
DGIn­dexDVD in­de­xing tool

About the Tools

Many pe­o­ple come to me and ask what do I use for transco­ding – Me­GUI? Za­rx­GUI? So­me­thing else? I don’t use any GUI. All you need for transco­ding is Avi­synth, x264.exe en­co­der and batch file to sup­ply pa­ra­me­trs to x264.exe. En­co­ding then runs in com­mand line. No­thing with GUI is ne­e­ded (well, ex­cept some sim­ple text edi­tor and Win­dows OS).


notepad.exe

Avi­synth is so called fra­me­ser­ving SW. It ta­kes fra­mes from vi­deo, pro­ces­ses them and sup­plies them to en­co­der. It doesn’t have any GUI, so don’t ex­pect any win­dows af­ter in­sta­lation. It runs on bac­kground and you’ll ne­ver see it. You con­t­rol it thru scripts, vi­deo pro­ces­sing is han­dled by plu­gins (fil­ters) in form of .dll lib­ra­ries.

x264.exe is en­co­der for h.264 co­dec, which is pro­ba­bly the most advan­ced co­dec you can get your hands on. It’s for free (ba­si­cally; read li­cence if you want to make mo­ney with it or so­me­thing). x264.exe has ex­tensive po­si­bi­li­ties of con­t­rol via pa­ra­me­ters but de­faults are sane and good, so you don’t need to change any­thing. Dis­cuss. (I’m not re­ally good with x264.exe pa­ra­me­t­res.)

You will need text edi­tor for cre­a­ting Avi­synth (AVS) scripts and batch fi­les.

DGIn­dex cre­a­tes in­dex file of fra­mes in DVD which allows Avi­synth to read the vob (MPEG‐2) fi­les. You can use it for se­lecting cer­tain parts of DVD (e.g. one epi­sod of se­ries), so you don’t have to en­code it whole. It also out­puts au­dio of se­lec­ted part.

En­co­ding Pro­cess

Step 1: Cre­a­ting AVS script

AVS script has to con­tain lo­a­ding used fil­ters, lo­a­ding source vi­deo file and com­mands how to pro­cess the vi­deo. Ba­sic fil­ters don’t have to be ma­nu­ally lo­a­ded so some mi­ni­ma­lis­tic script can look like this:

avisource("d:\encoding\source_video.avi")

Avi­source is name of plu­gin which lo­ads avi vi­deos. For more uni­ver­sal use, di­rect­show­source() and FF­Vi­deo­Source() are avai­la­ble. FF­Vi­deo­Source ne­eds to be lo­a­ded ma­nu­ally:

loadplugin("d:\encoding\AVS_filters\ffms2-r430\ffms2.dll")
FFVideosource("d:\encoding\source_video.avi")


AVSP­mod edi­tor

If you put all your plu­gins to plu­gin fol­der of Avi­synth, then you don’t have to load them ma­nu­aly. I don’t do it for cer­tain re­a­sons, you can.

MP4/MKV

Now, let’s say you have full HD raw anime vi­deo in mp4 con­ta­i­ner (can be mkv too, doesn’t make di­f­fe­rence) with h.264 co­dec that has 900 MB fi­le­size, and you need to make it smaller and 720p (720p TV rip should be be­low 350 MB per 24 mi­nu­tes) and you need to hard­sub Sub­stati­o­nAl­pha sub­tit­les (ASS) to it (so they’re „bur­ned“ in images). There is also some black bor­der you want to crop out from the vi­deo. The script would look like this:

loadplugin("d:\encoding\AVS_filters\vsfilter.dll")
directshowsource("d:\encoding\Anime_ep01[1080p].mp4", audio=false)
Crop(0, 4, -0, -4)
LanczosResize(1280,720)
textsub("d:\encoding\sub\Anime_ep01_hardsub.ass")

AVSP­mod edi­tor

This script lo­ads VSfil­ter, which pro­vi­des cer­tain other fil­ters (text­sub). Then it lo­ads vi­deo (wi­thout au­dio) using di­rect­show­source. Then it crops 4 pi­xels from it on the right and left. Then it re­si­zes the vi­deo using La­cz­sos re­size me­thod (for other me­thod see this page). Then it hard­subs the sub­tit­les into vi­deo. The pro­ces­sed fra­mes then go to en­co­der.

You can cre­ate such script now and drag&drop it to Vir­tu­al­Dub to test it.

DVD

Now ano­ther exam­ple. Let’s you have a DVD (in other words bunch of .vob fi­les) and you want to transcode them into mkv with h.264 co­dec.

First you cre­ate .d2v file (in­dex) in DGIn­dex. If you don’t know how to do this, re­fer to this gu­ide.

Now, be pre­pa­red that when you transcode DVDs you will need to do a lot of fil­te­ring. DVDs are usu­ally the worst crap you can find con­si­de­ring the qua­lity of image (ex­cept stre­ams). De­pen­ding on re­gion the DVD is from, it’ll have ei­ther 50 fieldes/second (25 full frames/second) or 60 fieldes/second (30 full frames/second). You will have to put these fiel­des to­ge­te­her to cre­ate full fra­mes. This brings need of de­in­ter­la­cing and other stuff, it’s bey­ond frame of this gu­ide to go into it more tho­rou­ghly. Your script ba­si­cally ne­eds to load d2v file, get rid of fiel­des, de­in­ter­lace and clean image. Com­pare the cle­a­ned (up­per) and raw (lower) scre­ens of DVD R1:


DVD image be­fore and af­ter pro­ces­sing

The script mi­ght look like this:

#in the 1st part of script we load plugins:
loadplugin("d:\encoding\AVS_filters\vsfilter.dll")
loadplugin("d:\encoding\AVS_filters\DGDecode.dll")
loadplugin("d:\encoding\AVS_filters\TDeint.dll")
loadplugin("d:\encoding\AVS_filters\ColorMatrix.dll")
loadplugin("d:\encoding\AVS_filters\TBilateral.dll")
loadplugin("d:\encoding\AVS_filters\TweakColor.dll")
loadplugin("d:\encoding\AVS_filters\MSharpen.dll")
loadplugin("d:\encoding\AVS_filters\TIVTC.dll")
loadplugin("d:\encoding\AVS_filters\RemoveGrain\RemoveGrain.dll")
loadplugin("d:\encoding\AVS_filters\WarpSharp.dll")
loadplugin("d:\encoding\AVS_filters\MaskTools-v1.5.8\MaskTools-v1.5.8\MaskTools.dll")
loadplugin("d:\encoding\AVS_filters\masktools-v2.0a48\mt_masktools-26.dll")
loadplugin("d:\encoding\AVS_filters\temporalcleaner_25_dll\TemporalCleaner.dll")
loadplugin("d:\encoding\AVS_filters\degrainmedian_20061008\degrainmedian.dll")
loadplugin("d:\encoding\AVS_filters\FluxSmooth-1.1a\FluxSmooth.dll")

#and in second part we process the image:
mpeg2source("d:\encoding\SomeDVD\SomeDVD_part01.d2v", info=3) tfm(d2v="d:\encoding\SomeDVD\SomeDVD_part01.d2v") ColorMatrix(hints=false, interlaced=true, threads=0) tdecimate(mode=1,hybrid=1) TDeint(order=1) DeGrainMedian(limitY=5,limitUV=5,mode=1) fluxsmoothST(4,4) MSharpen(strength=50) tweakcolor(sat=1.1, bright=-6, cont=1.4) TBilateral(diameterL=7, diameterC=3, sDevL=1.7, sDevC=1.7, iDevL=1.6, iDevC=5.4, csL=1.0, csC=1.3, d2=false, chroma=true, kernS=1, kernI=8, resType=0, gui=false) textsub("m:\Other\encoding\logoACX_v1.ass") #if you want to hardsub some logo in ASS subs or something

Well… it’s blo­ody long, isn’t it? If you want to un­der­stand it, study following:

Mpeg2Source
TFM, TDe­ci­mate
Co­lor­ma­t­rix
De­gra­in­Me­dian
flu­xsmo­o­thST – ge­ne­rally about smo­o­thers
MShar­pen
tweak­co­lor
TBi­la­te­ral – yeah, aw­some fil­ter… you can spend days on settings like I did and you’ll ne­ver dis­co­ver eve­ry­thing… of course there are ma­ybe bet­ter smo­o­thers, but I like this one.

Many fil­ters and re­fe­ren­ces for them can be found at Avi­synth wiki.

Here you can down­load most of the fil­ters I used in this script.

Blu­ray

En­co­ding BDs is co­ve­red in my other gu­ide here.

So that’s it for cre­a­ting AVS script. That was the har­dest part (al­though it’s actu­ally very sim­ple if you don’t start by en­co­ding DVDs). We can now move on to the actual en­co­ding.

Step 2: En­co­ding with x264.exe

As I said, x264 has CLI (com­mand line) in­ter­face, so any settings are pas­sed to it by pa­ra­me­ters. Syn­tax is like this:

x264.exe [options] --output outfile infile

To make it sim­plier, you can use batch file. To cre­ate one, open up notepad.exe and copy this:

x264.exe --crf 17 --preset slow --tune animation --output encoded_video.mp4 AVS_script.avs
pause

The pa­ra­me­ters above mean fo­lowing:

–crf – con­stant qua­lity mode (va­ri­a­ble bit­rate), lower num­ber me­ans higher qua­lity, sane range of va­lues is 15–25. Higher the re­so­lu­tion is, higher va­lues you can use. There is also –bit­rate op­tion sets con­stant bit­rate (in kB/s) – do not use this un­less you REALLY need pre­dictable fi­le­size!
–pre­set – con­t­rols speed/compression ratio – slower speed me­ans bet­ter com­pres­sion and smaller out­put file.
–tune – pre­setted tun­ning for film, ani­mation, grain, stil­li­mage, psnr, ssim, fast­de­code, ze­ro­la­tency. If you don’t re­ally fit into any of them, don’t use it at all.
encoded_video.mp4 is out­put file, AVS_script.avs is in­put file.
pause – that is com­mand for win­dows CMD to wait for a keystroke.

Now you have this line in your text edi­tor, save the file as encode.bat to the same fol­der where you have the x264.exe and AVS script. Then run it (as execu­table) and see what hap­pens.

If eve­ry­thing went how it should, you will see this:


x264.exe out­put af­ter suc­cess­full pro­cess

If not, you mi­ght en­coun­ter scre­ens like these.


x264.exe out­put af­ter en­co­ding fai­lure

Now, what mi­ght have gone wrong:

error in AVS script – you can be pretty sure that’s it. Open the Vir­tu­al­Dub, load the AVS file and see what it tells you. If you don’t un­der­stand it, go­o­gle the error, it’ll pro­ba­bly help you. Most com­mon errors are that plu­gin can­not be found (bad path), fil­ter does not exist (plu­gin not lo­a­ded) and bad syn­tax.
in­put fi­les are not where they’re sup­po­sed to be – be­fore you start fre­a­king out check the paths
Avi­synth is gi­ven so­me­thing it can’t read – aren’t you lo­a­ding some .vob file or so­me­thing which can’t be re­co­gni­zed by Avi­synth?
mo­vax – for eve­ry­thing what goes wrong blame mo­vax.

When you fix all the errors in your script, run the en­code and wait long enough, it will fi­nish and print out some sta­tis­tics. When you see this, it me­ans the en­code fi­nished:


x264.exe out­put on en­code fi­nish
Step 3: Au­dio

If you transcode whole fi­les (you don’t cut off some parts), it’s very easy with au­dio. De­mux the au­dio from the ori­gi­nal file and mux it with the transco­ded vi­deo. If you didn’t mess up with frame ra­tes, eve­ry­thing should be OK. In case you were en­co­ding DVD and you de­ci­ded that you want film fra­me­rate (24000/1001 fps), then you mi­ght en­coun­ter pro­blems with audio/video sync. However, this is again above the frame of this even now aw­fully long gu­ide.

Step 4: Sub­tit­les

If you have DVD and want to get text sub­tit­les, then use SubRip for rip­ping sub­tit­les from DVD and con­ver­ting them with OCR to text subs. As for how to work with SubRip, see this gu­ide (scroll down to the SubRip gu­ide).

Then you can style the sub­tit­les in Ae­gi­sub and mux them with vi­deo and au­dio.

Some advi­ces

It mi­ght not look like it, but there is much to mess when en­co­ding.

Con­stant bit­rate

Do not use con­stant bit­rate! It’s to­tal bull­shit­tery. There is no sane re­a­son to require pre­dictable file size and thus con­stant bit­rate. (No, no­body is bur­ning vi­deos to CDs any­more and no­body ca­res if the size is 233 MB or 258.4 MB.) With con­stant qua­lity you can achieve bet­ter qua­lity and smaller fi­les.


1: source image, 2: over­com­pres­sed image (view in full re­so­lu­tion)
Over­com­pres­sed vi­deo

Don’t be fag­got and check if you have enough high –crf va­lue set for your vi­deo. Each vi­deo requi­res so­me­thing sli­ghtly di­f­fe­rent, but ge­ne­rally you won’t mess up using –crf 17 for 480p and smaller vi­deo re­so­lu­tion, then raise it by one for each cca 200 px of he­i­ght of the vi­deo (very ge­ne­ral rule and mi­ght not ap­ply in some ca­ses).

Un­der­com­pres­sed vi­deo

As I wrote in my BD gu­ide, don’t be like THORA and Co­al­Gi­rls and such fan­sub groups, which have 2 hours 1080p mo­vie in 12 GB. For 2 hours anime mo­vie 5 GB is far enough even with grain sce­nes in half of it. (Live action mo­vies require sli­ghtly lower –crf va­lues and thus bi­g­ger fi­les, however 8 GB should be far enough too.) As for such mo­vie in 720p, 3 GB should be enough. Epi­sode of 24 mins should ex­ceed 600 MB and 400 MB re­specti­vely. (Yes, I am be­ing sli­ghtly in­con­si­s­tent with the BD gu­ide – the num­bers are very ap­pro­xi­mate.)

Flu­ency

The re­sul­ting vi­deo should play flu­ently wi­thout any fre­e­zing. There are many causes of fre­ezy vi­deo and I don’t even re­mem­ber what I have en­coun­te­red already. It is caused usu­ally by skip­ping every few fra­mes one frame or vice versa (du­pli­ca­ting fra­mes).

A/V sync

Ne­ver pro­duce file which has de­syn­chro­ni­zed au­dio and vi­deo tracks (and if you do sub­tit­les, then they have to be in per­fect sync too). This of­ten hap­pens when you transcode from one fra­me­rate (e.g. NTSC DVD, 30000/1001 fps) to ano­ther (e.g. film, 24000/1001 fps). You have to be ca­re­ful when do­ing this.

Here is an awesome exam­ple of vi­deo out of sync.

Hard­sub­bing

Do not hard­sub what doesn’t need to be hard­subbed. The only things you should hard­sub is ka­ra­oke and ma­ybe some small ino­btru­sive logo showing for few se­conds that you have en­co­ded it. Hard­sub­bing eats up bit­rate and it usu­ally sux since viewer can’t turn it off.


BD ups­ca­led from DVD
Vi­deo­size

Ne­ver ups­cale! But down­s­cale so­me­ti­mes – many BDs are DVD ups­ca­les (see image), so you can shar­pen it and down­s­cale it (in this or­der!) to 720p. Try not to do this if you feel you’d lo­ose some de­tails. Hint: very few anime have 1080p re­so­lu­tion, so you can down­s­cale to 720p quite sa­fely.

Don’t over­fil­ter it

Don’t ove­ruse fil­ters. Encoder’s task is to con­vert vi­deo to pretty watchable PC for­mat, not to cre­ate so­me­thing new. Keep this in mind espe­ci­ally when cle­a­ning DVD image. I my­self used to tend to make image su­per­clean and I usu­ally found af­terwords that I had cre­a­ted image so clean it was al­most de­tailless. Try to avoid that.

Thanks to ste­e­lista for this advice.

Other

Ter­mi­na­ting en­co­ding pro­cess – you can do so in the com­man­d­line win­dow by pres­sing ctrl‐C. There is no way how to pause it, all you can do is se­lect text in the win­dow (Win­dow Menu -> Edit -> Mark), this’ll pause the pro­cess. Un­pause it by pres­sing es­cape key.

ConvertToRGB32 – when you open a script in Vir­tu­al­Dub, you mi­ght en­coun­ter green or blue vi­deo screen in­stead of the picture. Try put­ting ConvertToRGB32() line at the end of your script to avoid this. You have to remove/comment this be­fore en­code.

Speed – be pre­pa­red that the en­code mi­ght take hours. De­pends on your CPU speed, of course. Also note, that Avi­synth is slow 32bit bitch, so if you have i7 CPU or so­me­thing as awesome as that, you’ll have to run a few en­co­des (I run usu­ally 4 on i7) at the same time to load it fully. These may share en­co­der (x264.exe file) and fil­ters but can­not share the AVS file. Also pay at­ten­tion to co­o­ling.

I’ll ma­ybe add stuff here.

Use­ful Tools

Ex­cept tools men­ti­o­ned above some of following mi­ght come in handy.

AVSP­mod – text edi­tor, sup­ports AVS syn­tax and inclu­des some tools like re­size cal­cu­la­tor.
MP4Box GUI – mux/demux MP4 fi­les
MKV Tool­nix – collection of tools for de­a­ling with MKV (ma­inly mu­xing)
MKV Ex­tractMKV de­mu­xer
MKV Clea­verMKV de­mu­xer with bet­ter batch sup­port

Use­ful Web­si­tes

There is lot of pages about transco­ding out there and be­lieve me, you will so­o­ner or la­ter read tons of text on them. There are va­ri­ous gu­i­des scat­te­red all over the In­ter­net, but I pic­ked the most helpful/important of them (in my opi­nion):

Avi­synth wiki – the only de­cently com­plex page about Avi­synth I know is this wiki; note the menu be­low the he­a­der – that’s the im­por­tant one
mewiki – con­ta­ins al­most eve­ry­thing about x264 en­co­der; I go there for settings de­scrip­tion if I need some of them
Vi­deo­help – actu­ally… why am I wri­ting this? Eve­ry­thing is there…

Epi­lo­gue

Not much to say here, but I always conclude! Well, so­me­thing about me – how I lear­ned transco­ding. One day I told a fri­end on IRC that I wanna know how to transcode. He sent me four li­nes of code. Some 3 li­nes of AVS script and 1 line of batch file. And that’s how I star­ted. Thanks, ste­e­lista. Then I lear­ned about fil­ters, for I ne­e­ded to transcode one aw­ful DVD. The en­co­des were aw­ful too, by the way… This was 3 years ago. En­co­ding is long run, I don’t con­si­der my­self as even good, not to say awesome guru lol (that doesn’t stop me from wri­ting these gu­i­des, though). So, have fun.

Chan­ge­log:

v1.05
v1.05: advice from ste­e­lista ad­ded
v1.04: ad­ded ab­so­lute path for plu­gins to make clear that they’re not lo­a­ded from Avi­synth de­fault plu­gins di­rec­tory

v1.03: some mi­nor chan­ges in in­tro­duction

2 thoughts on “How to start with transco­ding aka fai­rly sim­ple long ge­ne­ral transco­ding gu­ide

  1. I see your blog ne­eds some fresh ar­ticles. Wri­ting ma­nu­ally is time
    con­su­ming, but there is so­lu­tion for this. Just search for; Masquro’s
    stra­te­gies

  2. I have chec­ked your site and i’ve found some du­pli­cate
    con­tent, that’s why you don’t rank high in go­o­gle, but there
    is a tool that can help you to cre­ate 100% unique ar­ticles, search for; Boorfe’s
    tips un­li­mi­ted con­tent

Napsat komentář

Vaše emailová adresa nebude zveřejněna. Vyžadované informace jsou označeny *