I remember when Cycles first came out, people loved the progressive rendering, where it shows you the whole image and gets clearer and clearer the longer you wait. But one of the first things people asked was “Can you render an animation progressively too?“. The answer was no. Until now that is.
I’ve created an addon that allows you to do exactly that.
It’ll render each frame of the animation to its own folder (like path/to/render/frame_15/image.jpg), and then repeat the whole animation render again using a different seed, which means that there’ll be a different noise pattern. Then you can hit the Merge Seeds button and it’ll gather all the different images for each frame, average the pixel values and make a nice clean animation.
The idea of rendering different seeds and then merging them together is called Image Stacking. Yes I did a tut on this a while back, and yes the post has been deleted (due to the fact that I got it all wrong). Image stacking is simply taking a bunch of images with different noise patterns and mixing them all together, averaging their values. You can do this in Blender of course using the node editor by mixing the first two image with a Fac of 0.5, then the third image with the result of the last mix by 0.33, the fourth image with that result by 0.24, the fifth at 0.2… which is a lot of hard work if you have hundreds of seeds.
Cue ImageMagick! A small but powerful command line program that can do a bunch of stuff to images, like resizing them, converting them, aligning them, and oh yeah, merging them!
convert "path/to/images/*.png" -evaluate-sequence mean "path/to/images/merged_image.png"
That’s the command you’d use to merge a folder of images into a single image with less noise. If you like you can change mean to another mathematical function like median (which is apparently better when used in photography for merging high ISO noise images, but in my tests still proved worse)
That command is used in my addon, so make sure you have ImageMagick installed, otherwise it’ll moan at you.
Here’s a video overview of the addon, and below it is some temporary documentation before I finish it and put it on the wiki.
Repeats: The number of times to render each frame with a different seed. You can set this to some ridiculously unreachable number and then just stop it when you’re happy (you’ll have to kill the blender process to stop it though)
Samples per seed: Quite self-explanatory, the number of samples to render each time.
Output Path: This is exactly the same as the path in Blender’s usual Output panel, they are linked.
Folder Layout: You can either render to a bunch of folders where each frame has it’s own folder with the seed images inside those folders (/frame_15/seed_1.jpg, /frame_15/seed_2.jpg…) or have a folder for each seed set such that each of these folders has the entire animation in it(/seed_set_1/frame_15.jpg, /seed_set_1/frame_16.jpg). The latter (“Seeds”) is useful if you’d like to start compositing the animation before it has finished rendering all the seeds. This is probably better, but due to how ImageMagick works, a temporary folder is made when merging the images and all the images are copied into there, temporarily taking up disk space. Other than that there doesn’t seem to be any drawback that I can think of.
Seed Variance: A choice between incremental (increase the seed for every image) and random (a random seed between 0 and 1000000000). The only reason you’d need to use Incremental is if you want to know exactly what seeds you’ll get.
Overwrite: If enabled, the animation will render every frame for each seed just as you’d expect. But if you leave it disabled, it’ll skip images that have already been rendered. This effectively allows you to resume a stopped or crashed render. (It checks if the file exists and how many seeds have already been rendered for that frame)
Info: This block shows info like the total number of samples for each frame (samples per seed multiplied by repeats) and shows the paths to the images. When rendering or merging, this block will change to a progress indicator.
Merge on Completion: When all the seeds have been rendered, perform the Merge function.
Delete Seeds: When finished merging, delete the high-noise seed images, leaving just the merged ones.
Progressive Render/Animation: The big button. Render the current frame or full animation using the settings above.
Merge Seeds: Average the pixel values using ImageMagick. If you don’t have ImageMagick installed it’ll show an error and provide a link to the download page (and a link to this page on how to install it properly)
The end!
Please let me know in the comments if you have any ideas for a new feature, and report any bugs, crashes and errors.
Changelog:
- 26/07/2013: Added Overwrite option to allow resuming of unfinished renders
- 23/07/2013: Folder Layouts option, folders for seeds with each frame in them, or folders for frames with each seed in them.
I have to comment the second time in a row :) Great idea and great addon…will definitively give it a try and report back (maybe in IRC?) I’m doing the merge mostly with imagemagick but having all this in an easy to use one click solution is even better!
Greetings, Thomas
Great! I’ve still got some ideas for improvements, so stay tuned for updates!
Hi! Very nice stuff. My comment would be that it would be nicer if each *set of frames* went into separate dirs instead of creating dirs for each *frame’s samples*
Not only could this potentially create far less dirs (imagine a 1000 frame sequence rendered in 10 refinement steps) but also it is more suitable if you want to start editing or compositing your passes as soon as the first refinement step is over
Cheers!
Ah I see how that could be useful, I’ll add it as an option.
As it is now, you could start editing/comping the frames by merging them even while it’s busy (with another blender open to run it)
Thanks
Done! Just download it again and tell me if it all works as expected.
Hi! great addon! but I’ve tried it and I just noticed a huge gap.. it must work in Linux/MacOS! I have Imagemagick installed but the addon creates a bat file to merge the files, so it doesn’t work in any OS different from Windows..
Thanks for this great addon but.. I’ll hope for further development :D
Sparazza
Actually I did test it in Linux and made sure it’d work there too (don’t know about mac)
Linux can still execute bat files, as long as they are marked as executable (which is done)
Could you check the bat file’s permissions and tell me if it is indeed executable, and then try execute the bat file on it’s own and tell me what errors it shows?
Uops! my fault then! ^^ ‘
I do not know, however, what is the problem ..
Traceback (most recent call last):
File “/home/ddeluca/.config/blender/2.68/scripts/addons/progressive_animation_render.py”, line 107, in execute
f = open(scn.render.filepath+sep+”merge.bat”, ‘w+’)
FileNotFoundError: [Errno 2] No such file or directory: ‘//TestProgressive/merge.bat’
location: /usr/share/blender/2.68/scripts/modules/bpy/ops.py:188
The TestProgressive folder is inside my home, so Blender has write permissions
Thanks!
sparazza
Interesting. The ‘w+’ bit in that line is supposed to create the file if it doesn’t exist, so I think it’s because the folder doesn’t exist. Did you render an animation using the addon? There should be folders like //TestProgressive/somename1/image_#.png with images in it.
The //TestProgressive/frame_71/seed_#.png exists and is rendered in the proper way ( i see the progression rate in the dialog), but the merge.bat cannot be generated and of course the resulting merged png doesn’t appear… but i’ve managed to deal with this error:
I solved by using absolute path for the “Output Path”!
With the absolute path, the final .png is generated
Sparazza
Ah yes, that would make sense. I’ll test it out tonight and see if I can catch that error and automatically switch to non-relative paths. Thanks!
No problem and thanks for all your effort in this!
Sparazza
ive missed ya zaal
Hi greg!
I’m replying here because it’s way more likely you’ll see this here as opposed to the comments on my post…
Thanks for the great read, and you prompted me to go back and look at the results from using median vs. mean in my renders as opposed to photography (as in the article).
I think what is happening is that for a given pixel in a noisy area, the sample is an approximation of the shadow strength at that location. While median will do wonders for dampening spurious pixel values, it destroys the ramping of shadow strength that is built with each pass in cycles.
So median will produce a much more contrasted result. Mean blending (which is the correct way to go I realize after reading here and playing), does the more ‘correct’ process of averaging the pixel value at that location, returning a more ‘proper’ shadow strength result.
Thanks for bringing this to my attention! :)
Hey man
Yeah I usually try mean and median just in case, though I haven’t yet found a case of median being better – perhaps if you have only a couple images?
Hey Greg just stumbled across your work. Only had a quick look so far but I’m thinking this could tie in well with network rendering. Each slave can render X samples and return back the image. Using network render for one frame is something that isn’t working yet.
Exactly yeah. In fact, as long as you keep Overwrite disabled then you can do a progressive render on as many PCs as you like (assuming they save to the same folder) and nothing will be rendered twice, nor will you render too many seeds for each frame. Instant render farm :)
Hey, pixaal. Thought I’d bring this to your attention: I’m getting the same error that IM isn’t installed correctly, despite being in my path. I was thinking it was related to the fact that I installed the 16-bit version, but I doubt it. I also tried absolute versus relative pathing and that didn’t change it.
Also, can you make a way to dismiss the “IM is missing” error? Something like a confirmation or exit button would be good.
Figured out my own problem… I was trying to use multilayer 32-bit EXRs (my default settings), which of course won’t work with IM because it only supports HDR images when custom-built. The downloadable binaries do not have this enabled.
Also, are you getting bad merging when you try to merge PNGs with alpha? https://dl.dropboxusercontent.com/u/17480686/img_0.png
That’s the default cube with RGBA selected in the output dialogue. I’m getting this result with both 8-bit and 16-bit PNGs.
I get an error when it tries to merge the images. This is my output:
Property: bpy.context.scene.ProgAnimFolderStruct = ‘seeds’
Property: bpy.context.scene.ProgAnimSeedType = ‘incremental’
Info: Starting Progressive Animation Render…
Info: Progress: (20%) Rendering frame 1, seed 1
Info: Progress: (40%) Rendering frame 1, seed 2
Info: Progress: (60%) Rendering frame 1, seed 3
Info: Progress: (80%) Rendering frame 1, seed 4
Info: Progressive Render Complete!
Error: Traceback (most recent call last):
File “/home/zauber/.config/blender/2.68/scripts/addons/progressive_animation_render.py”, line 118, in execute
bpy.ops.progressive.merge_anim(animation=animation)
File “/home/zauber/Desktop/blender-2.68a-linux-glibc211-x86_64/2.68/scripts/modules/bpy/ops.py”, line 188, in __call__
ret = op_call(self.idname_py(), None, kw)
RuntimeError: Error: Traceback (most recent call last):
File “/home/zauber/.config/blender/2.68/scripts/addons/progressive_animation_render.py”, line 174, in execute
rmtree(scn.render.filepath)
File “/home/zauber/Desktop/blender-2.68a-linux-glibc211-x86_64/2.68/python/lib/python3.3/shutil.py”, line 449, in rmtree
_rmtree_safe_fd(fd, path, onerror)
File “/home/zauber/Desktop/blender-2.68a-linux-glibc211-x86_64/2.68/python/lib/python3.3/shutil.py”, line 395, in _rmtree_safe_fd
_rmtree_safe_fd(dirfd, fullname, onerror)
File “/home/zauber/Desktop/blender-2.68a-linux-glibc211-x86_64/2.68/python/lib/python3.3/shutil.py”, line 406, in _rmtree_safe_fd
onerror(os.unlink, fullname, sys.exc_info())
File “/home/zauber/Desktop/blender-2.68a-linux-glibc211-x86_64/2.68/python/lib/python3.3/shutil.py”, line 404, in _rmtree_safe_fd
os.unlink(name, dir_fd=topfd)
PermissionError: [Errno 1] Operation not permitted: ‘X0’
location: /home/zauber/Desktop/blender-2.68a-linux-glibc211-x86_64/2.68/scripts/modules/bpy/ops.py:188
location: :-1
Make sure your output path is an absolute path, not relative
I got it working, but a couple of problems:
* It always puts the output image in the directory above the output path. On Linux, it defaults to the /tmp/ path, and that extra trailing slash creates some issues (like, /tmp//frame_1/seed_1.png). When you remove the trailing slash, this results in it trying to output the image to the root directory, which of course fails.
* I created a subdirectory in /tmp for it to output to in order to workaround the above issue, but it deletes that subdirectory when done (instead of putting the images into the specified output directory).
* The merged image is empty, nothing but 100% transparent pixels.
I am using Linux ubuntu and had the same ‘transparent image’ as you mentioned. I solved it by adding ‘-alpha off’ in the command line. You can run this line in the terminal: convert *.png -alpha off -evaluate-sequence mean result_alphaOff.png
or edit the python file of the addon, search the keywork ‘mean’, then insert ‘-alpha off’ to the command line
-alpha off -evaluate-sequence mean
That appears to work, however the final image appears to darkened quite a bit, whereas the individual images are all the expected brightness.
Okay, I had removed the code to disable the alpha, and I’m not getting the blank image problem. However, I still have the issue with the image being excessively darkened.
Hello Greg. Would it be feasible to use Luminance HDR instead of ImageMagick to merge Cycles renderings? Thanks.
It’d be a better idea to just use compositor nodes to do it I think. Someday I’ll get back to this ;)
HI, is there a way to send network job using you script or networkjob that use you script ?
If you tell all the pc’s to render to the same folder (on a networked drive) then it should work fine. It won’t render more seeds than you specify, and it checks this by counting the files, so whether you use one pc or many, it should work the same.
Let me know if you have any problems or requests
just to be sure, I was trying to use the network renderer addon, it seems to just ignore your addon, right ?
Yes. My addon handles rendering on it’s own, it won’t work with the network render addon at all.
so when you write “tell your pc’s” you mean one by one :P
Thank you
Hello,
thanks a lot for your addon, but I am not able to install it.
I install it from the user preferences panel, install from file . But I cannot find it in the list of addons.
I got nothing in the render panel.
Blender 2.69
Win 7 64bits
i5 4670k
16GB
GTX 760 OC
Thanks for your help
Arnaud
Could you toggle the system console and try installing it again? It should show some kind of error
I double check in the console, nothing happens. I tried with exe and zip Blender.2.69 versions.
Just tried also with Blender-2.67-r57348-mingw64. Very strange behavior.
Thanks for your help.
Arnaud
That is weird – try with the 2.70 test build
I checked in the code of the file I downloaded, it was html coded.
I downloaded another time, everything is perfect.
I think dropbox messed something.
Bye
Arnaud
hey. maybe some one will take it as a usefull tip. Make sure that your render is set tu PNG format. I set it up as JPG and the merging operation failed. But when i switched to Png everything went smoothly. cheers.
It looks cool!
Is the purpose of this tool to have a rough animation to start to work on or it also save a good amount of render time overall?
Just to have something for the compositor to work with while the final clean render is still going. It won’t render any quicker.
Hi!
Nice addon, realy.
Did you saw this: http://cgcookie.com/blender/2014/07/31/rendering-single-frame-faster-multiple-computers/
It will be nice to add network rendering to your addon.
Thanks anyway and regards!
Thanks! Yeah I saw it, and it made me decide to improve (i.e. rewrite) this add-on :) But it’ll be a few months before it’s done…
Hi Greg, love the addon but have a little problem – can’t merge images with ImageMagick. I installed it twice, the test render from prompt works (the guy with the wand) but the addon can’t find it. Any way to set the path to ImageMagick manually? Or to do something else (install ImageMagick into specific directory) or any other trick? Thnx.
What error is shown? Is it the “ImageMagick might not be installed correctly…” thing?
Yes, that’s the one. Maybe because I installed 64bit version?
That error is shown if the imagemagick command fails for any reason. It could be that you don’t have permissions or something arbitrary like that. Take a look in the folder you were going to render to – there should be a merge.bat file with the imagemagick command in it – try running that manually (open a command prompt (search ‘cmd’ in the start menu) and drag the merge.bat file into it, press enter) and tell me what errors it shows then.
Warning! This script removes output directory that may contain final results, intermediate results and your important files (i.e. if you set it as D:\myproject\, you project may be removed by script)
hi, what about using the addon via command line?