Cameras from Photosynth

http://photosynth.net/

Cameras from Photosynth

Postby GregDowning » 19 Nov 2010 02:02

I have been extracting cameras from the Photosynth .json file. If you look at the info coming out of Photosynth you will find there are 4 values describing the camera's orientation. Photosynth is storing rotation in a quaternion format. Most 3D programs (ie Maya) use Euler values (simple degrees for 3 axis). I started w/ the code that user "art79" posted on the binarymillenium.com, then Simeon Basset cracked the nut on this conversion to help us texture an important artifact discovered while making a documentary on Egypt.

You can see an example of it in use here to create gigapixel textures: http://www.xrez.com/blog/photogrammetric-gigapixel-images/

Only problem we had is that each camera came flipped exactly 180 degrees from the correct orientation so maybe someone else can find that bug. We just flipped the cameras in Maya.

Here is the C# code used to extract the cameras:

Code: Select all
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO;
using NetServ.Net.Json;
using System.Diagnostics;




namespace json_parse
{

    class Program
    {

        static void Main(string[] args)
        {

            JsonObject jsonOb;
            JsonObject Joimage;

            JsonObject Jor;

            JsonObject Jogroup;

            string json = File.ReadAllText(@"0.json");

            using (JsonParser parser = new JsonParser(new StringReader(json), true))

                jsonOb = parser.ParseObject();

            JsonObject Jo1 = (JsonObject)jsonOb["l"];

            JsonObject Jo2 = (JsonObject)Jo1.First().Value;

            JsonObject Jox = (JsonObject)Jo2["x"];

            string CSVoutput = "group,image,p0,p1,p2,j0,j1,j2,j3,j4,j5,j6,j7,j8,eulerX,eulerY,eulerZ\n";

            foreach (string groupKey in Jox.Keys)
            {

                Jogroup = (JsonObject)Jox[groupKey];

                Jor = (JsonObject)Jogroup["r"];

                foreach (string ObKey in Jor.Keys)
                {

                    CSVoutput += groupKey + "," + ObKey;

                    Joimage = (JsonObject)Jor[ObKey];

                    foreach (JsonNumber coord in (JsonArray)Joimage["p"])

                        CSVoutput += "," + coord.Value.ToString();



                    // start simeon's changes

                    //int jCount = ((JsonArray)Joimage["j"]).Count;
                    //JsonNumber num;
                    //for (int i = 0; i <= 3; i++) {
                    //    num = (JsonNumber)((JsonArray)Joimage["j"])[i];
                    //    CSVoutput += "," + num.Value.ToString();
                    //}



                    //for (int i = 7; i <= 8; i++) {
                    //    num = (JsonNumber)((JsonArray)Joimage["j"])[i];
                    //    CSVoutput += "," + num.Value.ToString();
                    //}
                    //
                    // end simeon's changes

                    foreach (JsonNumber coord in (JsonArray)Joimage["j"])

                        CSVoutput += "," + coord.Value.ToString();

                    // start simeon's euler values
                    // use the quat.exe <float> <float> <float>
                    JsonNumber qNum;
                    qNum = (JsonNumber)((JsonArray)Joimage["j"])[4];
                    double qx = qNum.Value;
                    qNum = (JsonNumber)((JsonArray)Joimage["j"])[5];
                    double qy = qNum.Value;
                    qNum = (JsonNumber)((JsonArray)Joimage["j"])[6];
                    double qz = qNum.Value;

                    Process p = new Process();
                    p.StartInfo.UseShellExecute = false;
                    p.StartInfo.RedirectStandardOutput = true;
                    p.StartInfo.WorkingDirectory = "./";
                    p.StartInfo.FileName = "quat.exe";
                    p.StartInfo.Arguments = qx + " " + qy + " " + qz;
                    p.Start();
                    p.WaitForExit();
                    string eulerRot = p.StandardOutput.ReadToEnd();
                    CSVoutput += "," + eulerRot;
                    // end simeon's euler values

                    CSVoutput += "\n";

                }

            }

            File.WriteAllText(@"json.csv", CSVoutput);

        }



        static double[] unitQuaternionToXYZEuler(double qx, double qy, double qz)
        {

            double qw = Math.Sqrt(qx * qx + qy * qy + qz * qz);



            double[] q = { qx, qy, qz, qw };

            double xRot = Degrees(Math.Atan(

                (2 * (q[0] * q[3] + q[1] * q[2])) / (1 - (2 * (Math.Pow(q[2], 2) + Math.Pow(q[3], 2))))));

            double yRot = Degrees(Math.Asin(

                2 * (q[0] * q[2] - q[3] * q[1])));

            double zRot = Degrees(Math.Atan(

                (2 * (q[0] * q[1] + q[2] * q[3])) / (1 - (2 * (Math.Pow(q[1], 2) + Math.Pow(q[2], 2))))));



            double[] rot = { xRot, yRot, zRot };

            return rot;

        }



        static double Degrees(double rad)
        {

            return rad * (180 / Math.PI);

        }

    }

}
User avatar
GregDowning
 
Posts: 65
Joined: 10 Oct 2010 19:22

Re: Cameras from Photosynth

Postby Nathanael » 24 Nov 2010 23:35

Greg, fantastic work. I immediately thought of a request of Tom Benedict's when I saw the output on your web log.

Would I be correct to assume you were also aware of Christoph's camera parameter export to CSV in SynthExport? Would there be any value to contributing your code to SynthExport?
User avatar
Nathanael
Moderator
 
Posts: 58
Joined: 13 Oct 2010 01:34
Location: Salem: Oregon: Thulcandra

Re: Cameras from Photosynth

Postby Joshharle » 04 Dec 2010 05:38

Hi Guys,

I've just implemented a similar setup for 3DS Max on top of SynthExport; you can get it at http://blog.neonascent.net/archives/cameraexport-photosynth-to-camera-projection-in-3ds-max/

@Greg, in the end I didn't use the source you posted: it uses an external program called quat.exe to actually do the work, and not the two functions included in it. 3DS Max postitions cameras using transformation matrices too, so a bit of a different output needed. Thanks for the inspiring results! Can I ask what you used to pull the final render out into a multi-res format to view online?

Kind Regards,
Josh
Joshharle
 
Posts: 14
Joined: 27 Oct 2010 17:03

Re: Cameras from Photosynth

Postby Nathanael » 05 Dec 2010 20:39

Kean Walmsley also made a recent post on his web log about very similar work going on for the next version of Autodesk's Photo Scene Editor: Back from AU 2010

The future version of Photo Scene Editor (codenamed Caipi) whose results are shown is not yet publicly available, but Kean says it's only months away from release. I'll be interested to learn whether the next version is free or for charge.

For those who are new to the topic, tangentially related to this discussion by way of textured mesh generation, are ARC3D and hypr3D, though both of those products as well as Photo Scene Editor are not as open or under your own control as what Greg and Josh are offering.

EDIT: Has something happened to ARC3D? It's not loading for me today.
User avatar
Nathanael
Moderator
 
Posts: 58
Joined: 13 Oct 2010 01:34
Location: Salem: Oregon: Thulcandra

Re: Cameras from Photosynth

Postby Kjellis85 » 06 Dec 2010 09:12

The Caipi project seems very promising, at least in the texturing department (the mesh looks rough). But how will it compete to agisoft photoscan for instance? As I understand Caipi is a plug-in for autocad, no? And autocad alone costs between 3-4000 usd... But it does look promising :)

@Nathanael, Arc3D is buggy as hell, don't know if it's down right now since I haven't been using it in a while, but it goes down for periods of time with no apparent reason. But their support should answer relatively quick, at least they did with me.
AMD PHENOM II X6 1090T BE 3.2GHZ - 12GB CORSAIR XMS3 1600MHZ - SPINPOINT F3 1TB - CORSAIR FORCE 3 120GB - WINDOWS 7 HOME PREMIUM - ASUS GTX 570 DIRECTCU II 1280MB
User avatar
Kjellis85
 
Posts: 146
Joined: 09 Nov 2010 12:59
Location: Tromsø, Norway

Re: Cameras from Photosynth

Postby Kean » 06 Dec 2010 11:38

Actually not: it's a standalone product that's a new version of Photo Scene Editor, which is freely available while it's on Autodesk Labs, at least.

PSE allows you to manipulate "scene" in a way that just isn't possible in Photosynth - you can provide additional information to help images get included in the scene, you can orient & calibrate the scene for accurate measurement, you can remove points from the cloud/mesh etc. And the current granularity of the mesh is for performance in downstream apps - it can actually create a fine a mesh and you can stand.

@Nathanael - I'm curious what you mean by "under your control"... do you mean in terms of source code availability (which - last I checked - wasn't the case for Photosynth ;)) or the ability to control the results of the workflow (which is an area where PSE is actually *much* better that Photosynth)?

Cheers,

Kean
User avatar
Kean
 
Posts: 2
Joined: 06 Dec 2010 11:26

Re: Cameras from Photosynth

Postby Nathanael » 06 Dec 2010 14:55

Hi Kean,

Yes, I only meant in terms of being able to look at source and understand what is going on. You are right, of course, that Photosynth is just as closed as Photofly and for the same valid reason.

I had not meant to disparage Photo Scene Editor's interactive/editing abilities at all as there are not too many contenders for the features which it offers (if any).
User avatar
Nathanael
Moderator
 
Posts: 58
Joined: 13 Oct 2010 01:34
Location: Salem: Oregon: Thulcandra

Re: Cameras from Photosynth

Postby Joshharle » 06 Dec 2010 16:59

@Kean

I've just tried out the Photo Scene Editor, and was very impressed with the ability to manually fix matches. I see that the geometry is textured, but the level of detail in the geometry appears to just be based on initial feature points, not the dense pointcloud reconstruction you can get with PhotoSynth Toolkit or Bundler using PMVS. Am I missing something?

@Kjellis85 re: PhotoScan, I've found that PhotoSynth can sometimes be better at matching, and certainly the match times are not comparable: 20mins for photosynth versus 56 hours for photoscan to match images, and this is on a 6-core machine 2.8 Ghz machine. And of course PhotoSynth is free.

Kind Regards,
Josh
Joshharle
 
Posts: 14
Joined: 27 Oct 2010 17:03

Re: Cameras from Photosynth

Postby Kjellis85 » 06 Dec 2010 19:02

@Joshharle, I haven't had too much time to experiment with photoscan, but as you can see from these pictures the results are promising. I do however question your experience in the way that it took you 56(!) hours to complete a model that took 20 minutes in photosynth. My tests took somewhere around 1-2 hours, running on a i7-720QM with 8gb ram. It will inevitably take much longer time to compute on a private machine rather than in the cloud, but the number of points you get from photoscan far exceeds what you get from photosynth.

If you have some screencaps from your experiences I would love to see them :)
AMD PHENOM II X6 1090T BE 3.2GHZ - 12GB CORSAIR XMS3 1600MHZ - SPINPOINT F3 1TB - CORSAIR FORCE 3 120GB - WINDOWS 7 HOME PREMIUM - ASUS GTX 570 DIRECTCU II 1280MB
User avatar
Kjellis85
 
Posts: 146
Joined: 09 Nov 2010 12:59
Location: Tromsø, Norway

Re: Cameras from Photosynth

Postby Joshharle » 07 Dec 2010 05:34

@Kjellis85; I'm just trying to find a screen capture of the progress bar, but one of my reconstructions is here:

Here's some justification for the difference: A 300+ image set; PhotoSynth matches on a downsampled image; and PhotoScan will give a match for every camera. In contrast, PhotoSynth will give up if it doesn't get a good match.

I've noticed a pretty exponential increase in Alignment time based on imageset size with PhotoScan.

Kind Regards,
Josh
Joshharle
 
Posts: 14
Joined: 27 Oct 2010 17:03

Next

Return to Photosynth

Who is online

Users browsing this forum: No registered users and 1 guest