Peter Sinclair’s 3-D Conversions with LeiaPix demonstration was a big hit. These notes were provided by Peter.
The entire presentation, including a wide-ranging discussion about the future of stereo, is posted on YouTube: https://www.youtube.com/watch?v=7dg96elrvA8 .
How to Proceed
What we are going to do is start with an original 2D image, create a depth map (DM) using Leiapix. I use Leia now after using Midas Colab for a couple of years. Leia is easier to use, has support and is improving on a regular basis and generally gets decent results. It does cost something to use but that cost is not excessive (in my opinion).
For this workshop, I’m going to use an historic image that Mat sent me of the Palace Hotel in SF after the earthquake of 1906. Because this image is from the Library of Congress, there are high resolution versions of it available. I chose the 19.0 MB Tiff version from the Download Dropdown list. This gives you a 4962 W X 4002 size image which is a lot bigger than I usually use. I am going to time how long that it took to do each step and put in italics any comments that I have about a step.
Step 1 – Make the Original 2D Image as Good as Possible First
1: Crop the image roughly to get rid of unusable edges, resize to 2300 pixels W, rename it “Palace Hotel 1906” and make it a 12 quality jpg (I use Adobe Photoshop 2019)……………………………………………..1 minute.
2: We now need to clean up the image by cloning out the title and watermark on the pavement on the lower left, spot in the sky and telephone wires in the sky…………………….6 minutes…….( At this step I did not fix the telephone wires across the buildings. It takes a long time and is sometimes quite unnoticeable in the final conversion. It can always be done later)
3: We now need to crop the image to it’s useable size. We can clone to fix the edges which is time consuming, crop off the bad edges which is quick or do a combination of both. In this case, I decided to clone the left side which was quite quick and crop off the right side which would have been very slow to fix.(In layers you make a duplicate layer of the image, do all of your work on the duplicate and then flatten and save as a jpg if it all looks good. The reason you do this is so you don’t wreck your original while working on it)……………………………………….2 minutes
4: Finally we take the image and click Image/Auto Tone and Image/Auto Contrast. This might improve the image somewhat. Rename the finished image as “Palace Hotel 1906-orig”. Save as jpg………………………………………..20 seconds
After these steps we have our flat image at 2192 W X 1840 H and good enough to convert. From start to finish it took about 12 minutes to fix up the 2D photo.
Step 2 – Load the Original 2D Image into LeiaPix and Create Depth Map (DM)
1: Open Leiapix and drag and drop our finished “Palace Hotel 1906-orig” into the upload box. Choose Animation Style-Horizontal………Click Export/Next/Premium Export………..Use dropdown and choose “Depth Map (16 bit, up to 4K) 30”………..Click Save…..a Congrats message comes up.
Go to your Downloads Folder and the latest file will be “Palace Hotel 1906-orig _disparity.png”. Drag and Drop that into your working folder for the Palace Hotel 1906.
(To do all this costs me 30 credits with Leiapix but the DM does not have any watermarks on it….You can prebuy credits from Leia to use their services)
2: When the disparity.png is in your working folder, you will notice that white is in the foreground. I like to change it so black is in the foreground. To change it load the DM into Photoshop……Image/Adjustment/Invert/ Save…..Now black is foreground. You do that because SPM likes it that way. Just leave the file as .PNG.
Rename the DM it “Palace Hotel 1906-DM”. You now have your original 2D flat image and your Depth Map (Palace Hotel 1906-DM) for it identified. From now on these will be the versions that we use…………………………10 minutes
Step 3 – Load the Original + DM into Stereo Photo Maker and Evaluate the Quality of the Conversion
1: Open SPM (SterePhotoMaker) and drag and drop the orig and the DM into it. Make sure that the DM is on the right. (Use swap Left/Right command if the orig and DM are reversed). Click Alt + D and a dialog box comes up Put 60 in the deviation box and click F/R Position and with the pointer pick some people that are about 1/3 of the way into the scene (say 70?)… ( If you want everything behind the window put 0 in the F/R box). Click…..OK…………5 minutes
2: The converted image now comes up as SBS (side by side) pair. If you’re very lucky, the SBS may look great as it is and needs no fixups. If so, you just go to File/Save Stereo Image/ Name the file “Palace Hotel 1906-SBS”, Save it in the proper folder and you’re all done.
However, in most cases including this one, there are flaws that need to be fixed. Overall the buildings and left telephone poles look pretty good. But the people look a bit misplaced and a right side telephone pole needs straightening.
Step 4 – How to Fix the Quality of the Conversion
1: Study the SBS in SPM and see what needs fixing and what it will need. Leave the SBS in SPM so you can refer to it to see what needs fixing.
2: We open Photoshop and to do these fixes we need to layer the DM on top of the orig and name it “Palace Hotel 1906-mask”. Save it as a PSD file. We name the top layer “DM” and the bottom layer “orig”.
3: We are now going to add layers of things that need fixing and name them what they are (ie Right Pole)
4: With the DM layer highlighted, Go to layers dialog box and click little icon on top right which opens a dropdown list. Choose the top choice which is “Add new layer”. Name the new layer (right pole) and the continue adding new layers on top ( left boy) then (boy with papers) and so on.
5: On the “Palace Hotel 1906-mask” file, you click off the DM layer so that you can see the orig image. We are going to fix the “right Pole” first. Zoom in on the right pole so it is large in the picture.
6: Choose the “Lasso tool” (or any other selecting tool you may have) from the tool bar and outline the right pole. I start at the bottom left of the pole and click Alt and hold the left mouse and draw it all around the pole. Let go and you now have the pole outlined with dancing ants.
7: Now go to the top PS bar choose “Select/Save Selection…..Name as “right pole”……..OK.
8: Click on the DM eyeball again so that the dancing ants of the right pole are on the DM. Click on Ctrl + H so the ants disappear but the right pole is still selected. We are now going to choose the proper amount of gray to make the pole to the right depth.
9: We’re looking at a blurry gray mess, but we notice that the right pole is darker at the bottom than it is at top. Because dark is further to the front and lighter is further back, that means that the pole is leaning backward.
10: To make the pole stand up straight we need to make it the same gray color. The color we choose is at the bottom where the pole is attached to the ground. Choose the “Eyedropper tool” at 3 X 3 average and click at the bottom of the pole where it meets the ground. On the tool bar the foreground color box will become the color.
11: Choose the right pole layer in the layers list. From the top PS bar choose “Edit/Fill/ Contents are Foreground Color/ OK. The new colored pole will be in the right pole layer and is on top of the DM. You can click the right pole eyeball on and off to see what you’ve done.
12: We now go to the next layer above the “right pole” layer which is “left Boy”. You can click “H” to use the hand tool which lets you drag the image over to the left where the left boy is located. If we click the DM eyeball off and on we can see that the boys head does not have the same gray as the body so it needs fixing. You can also see the problems with the “boy with papers” which is the next layer to fix.
13: We now repeat Steps 6 through 11 on left boy. We see that there is a little bit of leakage of the boys body onto the DM from the original Leia conversion. To get rid of that, we go to the left boys outline and choose Select/ Inverse. We make a duplicate of the DM layer which is above the DM layer and it’s named “DM copy”. Choose the DM copy.
14: Use the “Clone Stamp tool” from the tool bar. Clone the matching background next to the boy to the edge of the boy’s body. You can do this quite roughly because the boy’s outline will prevent you for over cloning on his body. When finished click Select/Inverse to get outline back to the boy.
15: Repeat steps 6 through 14 on “boy with papers”. Use the same DM copy layers to make inverse changes. While doing this you can just clone out one small black mark on the ground near the boys leg and near his hat which does work. Correct this on the orig layer.
16: After these 3 steps we want to see how we are doing. On the “Palace Hotel 1906-mask” we turn on all eyeballs. On top PS bar File/Save as…..Name “Palace Hotel 1906-DM fix” and save as 12 quality jpg. Now turn off all eyeballs except for “orig” and File/Save as…. “Palace Hotel 1906-orig fix”.
17: In the folder that you are working in you should find these 2 new files. Load the DM fix and orig fix into SPM and make into an SBS. Look over the 3 parts that you fixed and see if you are OK with them. If not you can go back and make adjustments. For minor smearing I ignore until the finished DM is completed.
18: Every time you make a change and save the jpg put another number at the end of fix……ie: fix, fix1, fix2 and so on.
Step 5 – When the Conversion is Finished
1: When the SBS-3D image is satisfactory in SPM we now make it our final version.
2: It took 6 fixes to make the best DM. I’m showing you the “Palace Hotel 1906-mask” from the “Final version” folder. As you can see there are more layers that were added and fixed. Sometimes a layer was fixed more than once.
3: From the “Final version” folder you take the best DM which is “Palace Hotel 1906-DM fix6” and the best orig “Palace Hotel 1906-orig” and drag and drop into SPM and look at the SBS. If the SBS is good, you click “Z” to bring up the orig and DM in SPM. File/ Save Stereo Image……Name “Palace Hotel 1906-orig + DM fix6.jpg” in the folder.
4: Drag and drop the “Palace Hotel 1906-orig + DM fix6.jpg” back into SPM and nothing should change. That’s good it proved that you saved the right output.
5: You now do Alt + D and make it an SBS again. This time you can change the settings.. ie Deviation at 50 and F/R position at 0 (if you want everything to be behind the window). Save the SBS and name it “Palace Hotel 1906-SBS”. Look it over and we see some minor smearing in the SBS (the left image, right side along the bldg., right image right side of left boy)
6: It’s not too bad but if you wish to fix it, make a duplicate layer of the SBS and place on top as a PSD file. All corrections will be on the top layer. Use the clone tool and carefully fix up flaws that bother you. When you are finished look at the fixed SBS to see that you didn’t make any mistakes or retinal rivalries. If satisfied, save the PSD file as “Palace Hotel 1906-SBS fix”. Then flatten it and name “Palace Hotel 1906-SBS fix-final jpg”
7: To make an anaglyph, load the “Palace Hotel 1906-SBS fix-final.jpg” into SPM. Because our image is black and white, in SPM go to “Gray Anaglyph”/ red cyan and click. Anaglyph will appear. Look at it with your red/cyan glasses. Go to “Easy Adjustment” command on the tool bar and move the slider back and forth until you get the most pleasing effect. When achieved go to File/Save Stereo Image as “ Palace Hotel 1906-ana”.
Conclusion
This is the end of the presentation.
I welcome any constructive criticisms, suggestions, comments , how to improve, speed up or make easier any and all of methods that I use.
If there is a totally different and better way to do a 2D to 3D conversion I’d like to know.
I would also like to know an efficient way to do a gray scale map from a stereo pair.
I would like to know a better way to do selections (I have PS 2019)
I would like to know how to eliminate, reduce, correct, minimize or otherwise deal with smearing which seems to be prevalent in conversions done with depth maps.
I would be pleased if someone takes this image and runs with it to make a much better version than I did and then posts it and provides a workshop of the method.
-Peter Sinclair