Author Topic: Can I get an assist from Math Geniuses  (Read 10062 times)

0 Members and 1 Guest are viewing this topic.

nobody

  • Swamp Rat
  • Posts: 861
  • .net stuff
Can I get an assist from Math Geniuses
« on: March 20, 2013, 09:41:07 PM »
Posted in autolisp forum but someone suggested this would be better location. I agree.

I need to get estimated normal values for points extracted from a point cloud.  Specifically estimated from covariance analysis. The point cloud would be reduced by preprocessing methods as well as isolated to individual buildings.

What I'd like to do is demonstrated here:

http://pointclouds.org/documentation/tutorials/normal_estimation.php#normal-estimation

This is beyond me and my skills so I'm begging for anyone's help.

Thanks!

WILL HATCH

  • Bull Frog
  • Posts: 450
Re: Can I get an assist from Math Geniuses
« Reply #1 on: March 20, 2013, 10:46:52 PM »
I can help you next week if you don't manage anything before then.

Basically you need to figure out what that summation equation really means and do it for every point in the cloud.

It looks complicated, but really it's not too bad.  Very similar to doing audio effects on a DSP.

nobody

  • Swamp Rat
  • Posts: 861
  • .net stuff
Re: Can I get an assist from Math Geniuses
« Reply #2 on: March 21, 2013, 03:20:03 PM »
Thanks Will! I've been trying to find an alternative, but no luck. Will let you know.  I'd be happy to compensate you for your efforts. - Alien
« Last Edit: March 21, 2013, 10:14:41 PM by Alien »

WILL HATCH

  • Bull Frog
  • Posts: 450
Re: Can I get an assist from Math Geniuses
« Reply #3 on: March 24, 2013, 10:10:47 PM »
after a quick browse through AcPointCloud.h it appears there may be a function in place already for getting the normal vector.  Not quite sure yet if that's wuite what you're needing.

nobody

  • Swamp Rat
  • Posts: 861
  • .net stuff
Re: Can I get an assist from Math Geniuses
« Reply #4 on: March 25, 2013, 02:58:31 AM »
It could be.  The software I'm using is a piece of a beta project which uses xyz and n (n being normal vector info) to construct a building best as possible using aerial LIDAR data. That page was the one the gentleman who sent me the software directed me to.  I shot him an email to see if they developed an automated way to retrieve the "n" data.  I believe I downloaded the C++ (cpp) files and actual installation files for the program on that page. I'm trying to dig through and figure it out now.

I'm new to C#.Net but if accessing point cloud normal data is not different than accessing data for text or polylines, I should have no problem getting the info.  The important part really would be to insure the vector is aiming toward the centroid (inside) of the points... which I think is required to construct the model properly.

Thanks for the advice. 
« Last Edit: March 28, 2013, 10:29:16 PM by Alien »

WILL HATCH

  • Bull Frog
  • Posts: 450
Re: Can I get an assist from Math Geniuses
« Reply #5 on: March 25, 2013, 01:35:05 PM »
That said I'll look into what needs to be done to implement the needed Point Cloud Library materials.  Can you give me a point cloud to play with?

Thanks.

nobody

  • Swamp Rat
  • Posts: 861
  • .net stuff
Re: Can I get an assist from Math Geniuses
« Reply #6 on: March 26, 2013, 09:47:28 AM »
Definitely! I'm at work but the data I have is at home.  I'll upload this evening.

nobody

  • Swamp Rat
  • Posts: 861
  • .net stuff
Re: Can I get an assist from Math Geniuses
« Reply #7 on: March 27, 2013, 12:52:40 AM »
point cloud attached.

WILL HATCH

  • Bull Frog
  • Posts: 450
Re: Can I get an assist from Math Geniuses
« Reply #8 on: March 28, 2013, 01:54:05 AM »
Downloading the point cloud library stuff... damn it's almost 400 MB!

Sooo the point cloud you've uploaded is coming into my AutoCAD as an xref which is in your MyDocuments folder...
I have vanilla AutoCAD 2013, I've never worked with point clouds before so I don't know if this is something I need an object enabler to view
« Last Edit: March 28, 2013, 01:58:29 AM by WILL HATCH »

nobody

  • Swamp Rat
  • Posts: 861
  • .net stuff
Re: Can I get an assist from Math Geniuses
« Reply #9 on: March 28, 2013, 03:00:34 AM »
Crud! Sorry Will!  I believe point clouds are civil 3d / map objects... not sure if they created an object enabler for it.... without that, I don't think you can access the methods for it...  I'll try to see if I can't write something noobish this weekend that might access it ... if you don't mind I might be asking you for a little guidance if needed... is that okay? Hope I'm not waisting your time... I enjoy this stuff so it's basically a hobby for me

I did find they created the .exe to complete the normal vectors on that site I sent. I had to download a trillion things to get it... Unfortunately I have to convert the .las file (LIDAR file) to a .pcd file first (their own made up filetype), using another .exe they developed lol... the input needed though is technical, consisting of coordinates and related to the lidar flight paths, etc itself... a photogrammatrist would probably have no problems... unfortunately I am just a CAD geek... so I'm stuck having to dig in and learn about that lol... I'm scared that when I figure out how to convert to .pcd I'll have to do something else.... it's like a black hole......eventually it's gonna smoosh me lol

WILL HATCH

  • Bull Frog
  • Posts: 450
Re: Can I get an assist from Math Geniuses
« Reply #10 on: March 28, 2013, 03:02:58 AM »
After digging a little more in your drawing, it's almost entirely made up of unused block references.  The point cloud is just xref'd in.  Can you bind it, purge, and then upload again?

Playing with getting point cloud library set up... This is some very cool software!

nobody

  • Swamp Rat
  • Posts: 861
  • .net stuff
Re: Can I get an assist from Math Geniuses
« Reply #11 on: March 28, 2013, 10:18:58 PM »
Agreed! You can download almost any area/city through USGS website (.las file)...

I'm not sure how to bind my pointcloud...should be in my drawing (though it's a civil 3d drawing...with a bunch of stupid styles with blocks). I believe you are using the pointcloud part of autocad (pointcloudattach)... which I think is different than civil 3d pointclouds (see my attached)... but won't matter in the end... as long as there's a way to find the vectors...

If you're using the autocad version of attaching point clouds just need to type pointcloudattach and add this database (might have to dropdown your extensions to pick the right type).

Let me know. Thanks will!
« Last Edit: March 28, 2013, 10:27:25 PM by Alien »

nobody

  • Swamp Rat
  • Posts: 861
  • .net stuff
Re: Can I get an assist from Math Geniuses
« Reply #12 on: March 28, 2013, 10:23:47 PM »
I have developed one thought concerning this matter. I'm wondering if the centroid is totally necessary...I don't know.

If I simply pick a point in the middle somewhere, find the lowest point in the pointcloud (or just regular autocad points for that matter), go down an additional 10 meters or whatever, and find the vector from every point to it I think it would accomplish what I need...though I'm not sure because the vector needed is for "surface normals".... I don't understand that equation I posted but I the way the vectors are generated are actually by connecting 3 points, finding the vector perpendicular to the surface generated from those points toward the centroid, and then moving on... this would mean tons and tons and tons of calculations for even a small file. What do you think?

If I can't get  the pointcloud centroid to work I'll have to try the first option I mentioned... don't think i could pull off the second.
« Last Edit: March 28, 2013, 10:35:32 PM by Alien »

WILL HATCH

  • Bull Frog
  • Posts: 450
Re: Can I get an assist from Math Geniuses
« Reply #13 on: April 12, 2013, 11:03:29 AM »
I did some playing with the kd tree software gile recently posted (again MANY kudos gile)and managed to get something of a result.  It is quirky, and is by no means complete but could be something to get started.  Some of the points do not get a normal added.  I think this is due to a weak understanding of how to use the tree on my behalf.

Code - C#: [Select]
  1.         [CommandMethod("MyPointCloud")]
  2.         public void GeneratePoints()
  3.         {
  4.             Document doc = Application.DocumentManager.MdiActiveDocument;
  5.             Editor ed = doc.Editor;
  6.             Database db = doc.Database;
  7.             using (Transaction tr = doc.TransactionManager.StartTransaction())
  8.             {
  9.                 BlockTable bt = (BlockTable)tr.GetObject(db.BlockTableId, OpenMode.ForRead);
  10.                 BlockTableRecord btr = (BlockTableRecord)tr.GetObject(bt[BlockTableRecord.ModelSpace], OpenMode.ForWrite);
  11.                 Random rnd = new Random();
  12.                 for (int i = 0; i < 1000; i++)
  13.                 {
  14.                     DBPoint p = new DBPoint(new Point3d(5 * rnd.NextDouble(), 100 * rnd.NextDouble(), 100 * rnd.NextDouble()));
  15.                     btr.AppendEntity(p);
  16.                     tr.AddNewlyCreatedDBObject(p, true);
  17.                    
  18.                 }
  19.                 tr.Commit();
  20.             }
  21.         }
  22.         [CommandMethod("GetNormals")]
  23.         public void GetNormals()
  24.         {
  25.             Document doc = Application.DocumentManager.MdiActiveDocument;
  26.             Editor ed = doc.Editor;
  27.             Database db = doc.Database;
  28.             RXClass rxc = RXClass.GetClass(typeof(DBPoint));
  29.            
  30.  
  31.  
  32.                 using (BlockTableRecord btr = (BlockTableRecord)db.CurrentSpaceId.Open(OpenMode.ForWrite))
  33.                 {
  34.                     ObjectId[] ids = btr.Cast<ObjectId>().Where(id => id.ObjectClass == rxc).ToArray();
  35.                     int len = ids.Length;
  36.                     Point3d[] pts = new Point3d[len];
  37.                     for (int i = 0; i < len; i++)
  38.                     {
  39.                         using (DBPoint pt = (DBPoint)ids[i].Open(OpenMode.ForRead))
  40.                         {
  41.                             pts[i] = pt.Position;
  42.                         }
  43.                     }
  44.                     Point3dTree tree = new Point3dTree(pts, false);
  45.                     Dictionary<Point3d, List<Point3d>> neighbors = new Dictionary<Point3d, List<Point3d>>();
  46.                     getNeighbors(tree, tree.Root, 15, neighbors);
  47.                     List<Vector3d> vectors = new List<Vector3d>();
  48.                     foreach (var set in neighbors)
  49.                     {
  50.                         Vector3d ave = new Vector3d(0, 0, 0);
  51.                         if (set.Value.Count < 2) continue;
  52.                         Point3d pt = Nearest(set.Key,set.Value);
  53.                         int count = set.Value.Count;
  54.                         set.Value.Remove(pt);
  55.                         for (int i = set.Value.Count-1; i > -1; i--)
  56.                         {
  57.                             vectors.Add(set.Key.GetVectorTo(pt));
  58.                             pt = Nearest(pt, set.Value);
  59.                             set.Value.Remove(pt);
  60.                         }
  61.                         Vector3d tmp;
  62.                         for (int i = 0; i < vectors.Count-1; i++)
  63.                         {
  64.                             tmp = vectors[i].CrossProduct(vectors[i + 1]);
  65.                             if (ave.DotProduct(tmp) < 0)
  66.                                 ave -= tmp;
  67.                             else
  68.                                 ave += tmp;
  69.                         }
  70.                         tmp = vectors[0].CrossProduct(vectors[vectors.Count - 1]);
  71.                         if (ave.DotProduct(tmp) < 0)
  72.                             ave -= tmp;
  73.                         else
  74.                             ave += tmp;
  75.                         vectors.Clear();
  76.                         ave = ave.DivideBy(ave.Length / 10);
  77.                         using (Line line = new Line(set.Key,set.Key+ave))
  78.                         {
  79.                             btr.AppendEntity(line);
  80.                         }
  81.                     }
  82.                
  83.                
  84.             }
  85.         }
  86.         private Point3d Nearest(Point3d pt, List<Point3d> pts)
  87.         {
  88.             double best = double.MaxValue;
  89.             Point3d ret = new Point3d();
  90.             foreach (Point3d pnt in pts)
  91.             {
  92.                 double cur =pt.DistanceTo(pnt);
  93.                 if (cur<best)
  94.                 {
  95.                     best = cur;
  96.                     ret = pnt;
  97.                 }
  98.             }
  99.             return ret;
  100.         }
  101.         private void getNeighbors(Point3dTree tree, KdTreeNode<Point3d> node, double dist, Dictionary<Point3d, List<Point3d>> neighbors)
  102.         {
  103.             if (node == null) return;
  104.             node.Processed = true;
  105.             if (neighbors.ContainsKey(node.Value))//this is just a sanity check, MUST be removed for processing large items once method is proven
  106.             {
  107.                 Application.DocumentManager.MdiActiveDocument.Editor.WriteMessage("\nPoint processed again");
  108.             }
  109.             else
  110.             {
  111.                 List<Point3d> pts = new List<Point3d>();
  112.                 foreach (Point3d pt in tree.NearestNeighbours(node, dist))
  113.                 {
  114.                     pts.Add(pt);
  115.                 }
  116.                 neighbors.Add(node.Value, pts);
  117.             }
  118.             getNeighbors(tree, node.LeftChild, dist, neighbors);
  119.             getNeighbors(tree, node.RightChild, dist, neighbors);
  120.            
  121.         }
« Last Edit: April 12, 2013, 02:20:39 PM by WILL HATCH »

gile

  • Gator
  • Posts: 2507
  • Marseille, France
Re: Can I get an assist from Math Geniuses
« Reply #14 on: April 12, 2013, 12:38:17 PM »
Hi WILL HATCH,

It seems you haven't uderstand the advantages of a Kd Tree structure when searching neighbours.
The interest of Kd trees is to avoid searching points in portions of the space (whole sub-trees). At each recursive loop the coordinate of the current node corresponding to the current depth is compared to this of the point which neighbours are searched to see if the search will continue only in the left child node, only in the right one or in both.
If you search systematically in both, you don't need to use a kd tree, a linear search in a classic collection should be faster.
See my last attempt with some more methods.
Speaking English as a French Frog

WILL HATCH

  • Bull Frog
  • Posts: 450
Re: Can I get an assist from Math Geniuses
« Reply #15 on: April 12, 2013, 02:27:09 PM »
Thank you gile! That is the response I was hoping for.  I was struggling to find a good way to handle that situation without a strong understanding on using the tree. Being very tight on time to work on this, that was my 1 hour attempt.  With your input it will be easy to finish this.

WILL HATCH

  • Bull Frog
  • Posts: 450
Re: Can I get an assist from Math Geniuses
« Reply #16 on: April 16, 2013, 06:57:13 AM »
Does this pretty well point you in the right direction now?  This generates normals using the latest version of the kd tree and orients them towards the viewport like the point cloud library does.  Not sure why, but this works on my laptop for 100k points, but at 1M it shat a brick and threw an exception

Code - C#: [Select]
  1.         [CommandMethod("GetNormals")]
  2.         public void GetNormals()
  3.         {
  4.             Document doc = Application.DocumentManager.MdiActiveDocument;
  5.             Editor ed = doc.Editor;
  6.             Database db = doc.Database;
  7.             RXClass rxc = RXClass.GetClass(typeof(DBPoint));
  8.  
  9.  
  10.  
  11.             using (BlockTableRecord btr = (BlockTableRecord)db.CurrentSpaceId.Open(OpenMode.ForWrite))
  12.             {
  13.                 ObjectId[] ids = btr.Cast<ObjectId>().Where(id => id.ObjectClass == rxc).ToArray();
  14.                 int len = ids.Length;
  15.                 HashSet<Point3d> pts = new HashSet<Point3d>();
  16.                 for (int i = 0; i < len; i++)
  17.                 {
  18.                     using (DBPoint pt = (DBPoint)ids[i].Open(OpenMode.ForRead))
  19.                     {
  20.                         pts.Add(pt.Position);
  21.                     }
  22.                 }
  23.                 Point3dTree tree = new Point3dTree(pts, false);
  24.                 ConcurrentBag<Tuple<Point3d, Vector3d>> normals = new ConcurrentBag<Tuple<Point3d, Vector3d>>();
  25.                 Vector3d camera = ed.GetCurrentView().ViewDirection;
  26.                 Parallel.ForEach(pts, pt =>
  27.                 {
  28.                     List<Point3d> neighborinos;
  29.                     neighborinos = tree.NearestNeighbours(pt, (int)5);
  30.                     Point3d npt = pt.Nearest(neighborinos), lpt;
  31.                     Vector3d ave = new Vector3d(0, 0, 0), tmp;
  32.                     int count = neighborinos.Count;
  33.                     while (neighborinos.Count > 1)
  34.                     {
  35.                         lpt = npt;
  36.                         neighborinos.Remove(npt);
  37.                         npt = npt.Nearest(neighborinos);
  38.                         tmp = pt.to(lpt).cross(pt.to(npt));
  39.                         if (ave.dot(tmp) < 0)
  40.                             ave -= tmp;
  41.                         else
  42.                             ave += tmp;
  43.                     }
  44.                     if (ave.dot(camera) < 0)
  45.                     {
  46.                         ave *= -1;
  47.                     }
  48.                     ave = ave / ave.Length;
  49.                     normals.Add(new Tuple<Point3d, Vector3d>(pt, ave));
  50.  
  51.                 });
  52.                 foreach (var item in normals)
  53.                 {
  54.                     using (Line line = new Line(item.Item1, item.Item1 + 200 * item.Item2))
  55.                     {
  56.                         btr.AppendEntity(line);
  57.                     }
  58.                 }
  59.             }
  60.         }

The routine requires the following extension methods to avoid making any api calls during parallel execution
Code - C#: [Select]
  1.     public static class NormalExtensions
  2.     {
  3.         public static Vector3d to(this Point3d a, Point3d b)
  4.         {
  5.             return new Vector3d(b.X - a.X, b.Y - a.Y, b.Z - a.Z);
  6.         }
  7.         public static Vector3d cross(this Vector3d a, Vector3d b)
  8.         {
  9.             return new Vector3d(a.Y * b.Z - a.Z * b.Y, a.Z * b.X - a.X * b.Z, a.X * b.Y - a.Y * b.X);
  10.         }
  11.         public static double dot(this Vector3d v1, Vector3d v2)
  12.         {
  13.             return v1.X * v2.X + v1.Y * v2.Y + v1.Z * v2.Z;
  14.         }
  15.         public static Point3d Nearest(this Point3d pt, List<Point3d> pts)
  16.         {
  17.             double best = double.MaxValue;
  18.             Point3d ret = new Point3d();
  19.             foreach (Point3d pnt in pts)
  20.             {
  21.                 double cur = pt.DistanceTo(pnt);
  22.                 if (cur < best)
  23.                 {
  24.                     best = cur;
  25.                     ret = pnt;
  26.                 }
  27.             }
  28.             return ret;
  29.         }
  30.     }
« Last Edit: April 16, 2013, 11:32:15 AM by WILL HATCH »

gile

  • Gator
  • Posts: 2507
  • Marseille, France
Re: Can I get an assist from Math Geniuses
« Reply #17 on: April 16, 2013, 02:07:56 PM »
Hi,

You do not need these extension methods, you can use the Vector3d/Point3d ones even in parallel execution because Vector3d, Point3d, ... are immutable strucures and the members of these structures are independent from AutoCAD.
Speaking English as a French Frog

WILL HATCH

  • Bull Frog
  • Posts: 450
Re: Can I get an assist from Math Geniuses
« Reply #18 on: April 16, 2013, 03:34:21 PM »
I was getting a fatal error when calling Vector3d.CrossProduct(Vector3d) and creating the extension fixed that problem.

gile

  • Gator
  • Posts: 2507
  • Marseille, France
Re: Can I get an assist from Math Geniuses
« Reply #19 on: April 17, 2013, 05:06:29 AM »
Hi,

From your code, this works fine for me using the Vector3d built-in methods

Code - C#: [Select]
  1.         [CommandMethod("GetNormals")]
  2.         public void GetNormals()
  3.         {
  4.             Document doc = Application.DocumentManager.MdiActiveDocument;
  5.             Editor ed = doc.Editor;
  6.             Database db = doc.Database;
  7.             RXClass rxc = RXClass.GetClass(typeof(DBPoint));
  8.  
  9.             using (BlockTableRecord btr = (BlockTableRecord)db.CurrentSpaceId.Open(OpenMode.ForWrite))
  10.             {
  11.                 ObjectId[] ids = btr.Cast<ObjectId>().Where(id => id.ObjectClass == rxc).ToArray();
  12.                 int len = ids.Length;
  13.                 HashSet<Point3d> pts = new HashSet<Point3d>();
  14.                 for (int i = 0; i < len; i++)
  15.                 {
  16.                     using (DBPoint pt = (DBPoint)ids[i].Open(OpenMode.ForRead))
  17.                     {
  18.                         pts.Add(pt.Position);
  19.                     }
  20.                 }
  21.                 Point3dTree tree = new Point3dTree(pts, false);
  22.                 Vector3d camera = ed.GetCurrentView().ViewDirection;
  23.  
  24.                 ParallelQuery<Tuple<Point3d, Vector3d>> normals =
  25.                     pts.AsParallel()
  26.                     .Select(pt => new Tuple<Point3d, Vector3d>(pt, GetNormal(pt, tree, camera)));
  27.  
  28.                 foreach (var item in normals)
  29.                 {
  30.                     using (Line line = new Line(item.Item1, item.Item1 + 200 * item.Item2))
  31.                     {
  32.                         btr.AppendEntity(line);
  33.                     }
  34.                 }
  35.             }
  36.         }
  37.  
  38.         private Vector3d GetNormal(Point3d pt, Point3dTree tree, Vector3d camera)
  39.         {
  40.             Vector3d normal = new Vector3d();
  41.             Vector3d[] vecs = tree.NearestNeighbours(pt, (int)5)
  42.                 .Cast<Point3d>()
  43.                 .Select(p => p - pt)
  44.                 .ToArray();
  45.             for (int i = 0; i < vecs.Length - 1; i++)
  46.             {
  47.                 Vector3d tmp = vecs[i].CrossProduct(vecs[i + 1]);
  48.                 normal = normal.DotProduct(tmp) < 0 ? normal - tmp : normal + tmp;
  49.             }
  50.             normal = normal / normal.Length;
  51.             return normal.DotProduct(camera) < 0 ? normal.Negate() : normal;
  52.         }
Speaking English as a French Frog

WILL HATCH

  • Bull Frog
  • Posts: 450
Re: Can I get an assist from Math Geniuses
« Reply #20 on: April 17, 2013, 01:27:15 PM »
Thanks gile!
Something I think is important to note here is that when iterating through the neighbors it should be done by grabbing the nearest in the group as I have done.  Reason being is this approximates creating triangular faces and averaging them together weighted by face area.  This is because vector cross product a x b = |a|*|b|*sin(angle between).

Also, consider the array of vectors {a:(1,0,0),b:(-1,0,0),c:(0,1,0),d:(0,-1,0)}
If they were evaluated linearly the only non-zero result would be b x c

gile

  • Gator
  • Posts: 2507
  • Marseille, France
Re: Can I get an assist from Math Geniuses
« Reply #21 on: April 17, 2013, 01:44:11 PM »
Hi,
I didn't care about the normal calculation algorithm. I only wanted to confirm and show an example of what I said about using Point3d and Vector3d structures in a parallel execution.
Speaking English as a French Frog

WILL HATCH

  • Bull Frog
  • Posts: 450
Re: Can I get an assist from Math Geniuses
« Reply #22 on: April 17, 2013, 02:38:15 PM »
That was more for anybody using this code for normal calculations.  Again, thank you for the clarification.  Your code is much cleaner than mine and you've shown me some new methods to investigate.

nobody

  • Swamp Rat
  • Posts: 861
  • .net stuff
Re: Can I get an assist from Math Geniuses
« Reply #23 on: April 18, 2013, 01:03:33 AM »
Thanks will! Can you please tell me what files you referenced?  Really appreciate your work with.

WILL HATCH

  • Bull Frog
  • Posts: 450
Re: Can I get an assist from Math Geniuses
« Reply #24 on: April 18, 2013, 01:41:46 AM »
Have many in the class there (rightly named random experiments) but I think the ones you need are:

using System;
using System.Collections.Generic;
using System.Collections.Concurrent;
using System.Linq;
using System.Threading.Tasks;
using Autodesk.AutoCAD.ApplicationServices;
using Autodesk.AutoCAD.Runtime;
using Autodesk.AutoCAD.DatabaseServices;
using Autodesk.AutoCAD.EditorInput;
using Autodesk.AutoCAD.Geometry;

and of course gile's kdtree.

keep me posted on how this goes plugging into other application, we would have use for this here too.

nobody

  • Swamp Rat
  • Posts: 861
  • .net stuff
Re: Can I get an assist from Math Geniuses
« Reply #25 on: April 18, 2013, 02:05:26 AM »
Will do!  I am getting one error when piecing everything together... I probably did it wrong:

Error   3   Cannot implicitly convert type 'Autodesk.AutoCAD.Geometry.Point3dCollection' to 'System.Collections.Generic.List<Autodesk.AutoCAD.Geometry.Point3d>'   C:\Users\TheCadExpert\Desktop\TestNormalVectors\NormalVectors\NormalVectors\myCommands.cs   49   36   NormalVectors





WILL HATCH

  • Bull Frog
  • Posts: 450
Re: Can I get an assist from Math Geniuses
« Reply #26 on: April 18, 2013, 02:10:31 AM »
guess I should have mentioned I modded gile's tree there to return a List... :oops:
Code - C#: [Select]
  1.         public List<Point3d> NearestNeighbours(Point3d point, int number)
  2.         {
  3.             List<Tuple<double, Point3d>> pairs = new List<Tuple<double, Point3d>>(number);
  4.             this.center = point;
  5.             if (ignoreZ)
  6.             {
  7.                 this.center2d = point.Flatten();
  8.                 this.distanceTo = Distance2dTo;
  9.             }
  10.             else
  11.             {
  12.                 this.distanceTo = Distance3dTo;
  13.             }
  14.             GetNNeighbours(number, double.MaxValue, this.Root, pairs);
  15.             List<Point3d> points = new List<Point3d>();
  16.             for (int i = 0; i < pairs.Count; i++)
  17.             {
  18.                 points.Add(pairs[i].Item2);
  19.             }
  20.             return points;
  21.         }

gile

  • Gator
  • Posts: 2507
  • Marseille, France
Re: Can I get an assist from Math Geniuses
« Reply #27 on: April 18, 2013, 02:27:50 AM »
Hi,

You can try this way, without changing anything to the Point3dTree class.
Note I changed the GetNormal() method to follow WILL HATCH advice in reply #20.

Code - C#: [Select]
  1.         [CommandMethod("GetNormals")]
  2.         public void GetNormals()
  3.         {
  4.             Document doc = Application.DocumentManager.MdiActiveDocument;
  5.             Editor ed = doc.Editor;
  6.             Database db = doc.Database;
  7.             RXClass rxc = RXClass.GetClass(typeof(DBPoint));
  8.  
  9.             using (BlockTableRecord btr = (BlockTableRecord)db.CurrentSpaceId.Open(OpenMode.ForWrite))
  10.             {
  11.                 ObjectId[] ids = btr.Cast<ObjectId>().Where(id => id.ObjectClass == rxc).ToArray();
  12.                 int len = ids.Length;
  13.                 HashSet<Point3d> pts = new HashSet<Point3d>();
  14.                 for (int i = 0; i < len; i++)
  15.                 {
  16.                     using (DBPoint pt = (DBPoint)ids[i].Open(OpenMode.ForRead))
  17.                     {
  18.                         pts.Add(pt.Position);
  19.                     }
  20.                 }
  21.                 Point3dTree tree = new Point3dTree(pts, false);
  22.                 Vector3d camera = ed.GetCurrentView().ViewDirection;
  23.  
  24.                 ParallelQuery<Tuple<Point3d, Vector3d>> normals =
  25.                     pts.AsParallel()
  26.                     .Select(pt => new Tuple<Point3d, Vector3d>(pt, GetNormal(pt, tree, camera)));
  27.  
  28.                 foreach (var item in normals)
  29.                 {
  30.                     using (Line line = new Line(item.Item1, item.Item1 + 200 * item.Item2))
  31.                     {
  32.                         btr.AppendEntity(line);
  33.                     }
  34.                 }
  35.             }
  36.         }
  37.  
  38.         private Vector3d GetNormal(Point3d pt, Point3dTree tree, Vector3d camera)
  39.         {
  40.             Point3dCollection neighbors = tree.NearestNeighbours(pt, 5);
  41.             Point3d npt = Nearest(pt, neighbors), lpt;
  42.             Vector3d normal = new Vector3d(), tmp;
  43.             while (neighbors.Count > 1)
  44.             {
  45.                 lpt = npt;
  46.                 neighbors.Remove(npt);
  47.                 npt = Nearest(npt, neighbors);
  48.                 tmp = (lpt - pt).CrossProduct(npt - pt);
  49.                 if (normal.DotProduct(tmp) < 0)
  50.                     normal -= tmp;
  51.                 else
  52.                     normal += tmp;
  53.             }
  54.             if (normal.DotProduct(camera) < 0)
  55.             {
  56.                 normal = normal.Negate();
  57.             }
  58.             return normal / normal.Length;
  59.         }
  60.  
  61.         private Point3d Nearest(Point3d pt, Point3dCollection pts)
  62.         {
  63.             double best = double.MaxValue;
  64.             Point3d ret = new Point3d();
  65.             foreach (Point3d pnt in pts)
  66.             {
  67.                 double cur = pt.DistanceTo(pnt);
  68.                 if (cur < best)
  69.                 {
  70.                     best = cur;
  71.                     ret = pnt;
  72.                 }
  73.             }
  74.             return ret;
  75.         }
« Last Edit: April 18, 2013, 05:27:40 AM by gile »
Speaking English as a French Frog

WILL HATCH

  • Bull Frog
  • Posts: 450
Re: Can I get an assist from Math Geniuses
« Reply #28 on: April 18, 2013, 05:29:29 AM »
gile,

I haven't gotten into the world of linq yet, but this has been a good learning experience for me! Not understanding how ParallelQuery works I thought to do the following with the hope of catching up to your method in execution time by not waiting for the vector calculations to finish before starting to draw lines.  But eventually discovered that ParallelQuery is doing the same thing, only slightly faster.  Thanks for the great examples!
Code - C#: [Select]
  1.         [CommandMethod("cGetNormals")]
  2.         public void GetNormals2()
  3.         {
  4.             Document doc = Application.DocumentManager.MdiActiveDocument;
  5.             Editor ed = doc.Editor;
  6.             Database db = doc.Database;
  7.             RXClass rxc = RXClass.GetClass(typeof(DBPoint));
  8.  
  9.  
  10.             System.Diagnostics.Stopwatch tmr = new System.Diagnostics.Stopwatch();
  11.             tmr.Start();
  12.             using (BlockTableRecord btr = (BlockTableRecord)db.CurrentSpaceId.Open(OpenMode.ForWrite))
  13.             {
  14.                 ObjectId[] ids = btr.Cast<ObjectId>().Where(id => id.ObjectClass == rxc).ToArray();
  15.                 int len = ids.Length;
  16.                 HashSet<Point3d> pts = new HashSet<Point3d>();
  17.                 for (int i = 0; i < len; i++)
  18.                 {
  19.                     using (DBPoint pt = (DBPoint)ids[i].Open(OpenMode.ForRead))
  20.                     {
  21.                         pts.Add(pt.Position);
  22.                     }
  23.                 }
  24.                 ConcurrentQueue<Tuple<Point3d, Vector3d>> normals = new ConcurrentQueue<Tuple<Point3d, Vector3d>>();
  25.                 Vector3d camera = ed.GetCurrentView().ViewDirection;
  26.                 workerThread wt = new workerThread(pts, normals, camera);
  27.                 Thread worker = new Thread(new ThreadStart(wt.doWork));
  28.                 worker.Start();
  29.                 Tuple<Point3d, Vector3d> point;
  30.                 bool dequeued = false;
  31.                 while (!worker.IsAlive) ;
  32.                 while (worker.IsAlive || dequeued)
  33.                 {
  34.                     if (dequeued = normals.TryDequeue(out point))
  35.                     {
  36.                         using (Line line = new Line(point.Item1, point.Item1 + 200 * point.Item2))
  37.                         {
  38.                             btr.AppendEntity(line);
  39.                         }
  40.                     }
  41.                 }
  42.             }
  43.             tmr.Stop();
  44.             ed.WriteMessage("\nElapsed: {0}", tmr.ElapsedTicks);
  45.         }


Code - C#: [Select]
  1.     public class workerThread
  2.     {
  3.         HashSet<Point3d> pts;
  4.         ConcurrentQueue<Tuple<Point3d, Vector3d>> normals;
  5.         Point3dTree tree;
  6.         Vector3d camera;
  7.         public workerThread(HashSet<Point3d> p, ConcurrentQueue<Tuple<Point3d, Vector3d>> n, Vector3d c)
  8.         {
  9.             pts = p;
  10.             normals = n;
  11.             tree = new Point3dTree(pts, false);
  12.             camera = c;
  13.         }
  14.         public void doWork()
  15.         {
  16.             Parallel.ForEach(pts, pt =>
  17.             {
  18.                 List<Point3d> neighborinos = tree.NearestNeighbours(pt, (int)5);
  19.                 Point3d npt = pt.Nearest(neighborinos), lpt;
  20.                 Vector3d ave = new Vector3d(0, 0, 0), tmp, lvt = pt.to(npt);
  21.  
  22.                 while (neighborinos.Count > 1)
  23.                 {
  24.                     lpt = npt;
  25.                     neighborinos.Remove(npt);
  26.                     npt = npt.Nearest(neighborinos);
  27.                     tmp = lvt.cross(lvt = pt.GetVectorTo(npt));
  28.                     if (ave.dot(tmp) < 0)
  29.                         ave -= tmp;
  30.                     else
  31.                         ave += tmp;
  32.                 }
  33.  
  34.                 if (ave.dot(camera) < 0)
  35.                 {
  36.                     ave *= -1;
  37.                 }
  38.                 ave = ave / ave.Length;
  39.                 normals.Enqueue(new Tuple<Point3d, Vector3d>(pt, ave));
  40.  
  41.             });
  42.         }
  43.     }

WILL HATCH

  • Bull Frog
  • Posts: 450
Re: Can I get an assist from Math Geniuses
« Reply #29 on: April 18, 2013, 05:38:57 AM »
Also, out of curiosity I compared the extensions I created to the API methods.  I suspected mine would be slightly faster since I was avoiding wrapper creation through calling the native code.  Any thoughts on why this is not the case for the dot product?

(this is working on a 100k point cloud)

My VectorTo Elapsed: 7118
Ac VectorTo Elapsed: 8509
My Dot Elapsed: 4590
Ac Dot Elapsed: 2371
My Cross Elapsed: 9527
Ac Cross Elapsed: 10009


Code - C#: [Select]
  1.         [CommandMethod("VectorSpeed")]
  2.         public void MeasureSpeed()
  3.         {
  4.             Document doc = Application.DocumentManager.MdiActiveDocument;
  5.             Editor ed = doc.Editor;
  6.             Database db = doc.Database;
  7.             RXClass rxc = RXClass.GetClass(typeof(DBPoint));
  8.  
  9.  
  10.             System.Diagnostics.Stopwatch tmr = new System.Diagnostics.Stopwatch();
  11.  
  12.             using (BlockTableRecord btr = (BlockTableRecord)db.CurrentSpaceId.Open(OpenMode.ForWrite))
  13.             {
  14.                 ObjectId[] ids = btr.Cast<ObjectId>().Where(id => id.ObjectClass == rxc).ToArray();
  15.                 int len = ids.Length;
  16.                 HashSet<Point3d> pts = new HashSet<Point3d>();
  17.                 for (int i = 0; i < len; i++)
  18.                 {
  19.                     using (DBPoint pt = (DBPoint)ids[i].Open(OpenMode.ForRead))
  20.                     {
  21.                         pts.Add(pt.Position);
  22.                     }
  23.                 }
  24.                 Point3d[] points = pts.ToArray();
  25.                 HashSet<Vector3d> vecs = new HashSet<Vector3d>();
  26.                 for (int i = 0; i < points.Length - 1; i++)
  27.                 {
  28.                     vecs.Add(points[i].to(points[i + 1]));
  29.                 }
  30.                 Vector3d[] vtrs = vecs.ToArray();
  31.                 tmr.Start();
  32.                 for (int i = 0; i < points.Length - 1; i++)
  33.                 {
  34.                     points[i].to(points[i + 1]);
  35.                 }
  36.                 tmr.Stop();
  37.                 ed.WriteMessage("\nMy VectorTo Elapsed: {0}", tmr.ElapsedTicks);
  38.                 tmr.Restart();
  39.                 for (int i = 0; i < vtrs.Length - 1; i++)
  40.                 {
  41.                     vtrs[i].cross(vtrs[i + 1]);
  42.                 }
  43.                 tmr.Stop();
  44.                 ed.WriteMessage("\nMy Cross Elapsed: {0}", tmr.ElapsedTicks);
  45.                 tmr.Restart();
  46.                 for (int i = 0; i < vtrs.Length - 1; i++)
  47.                 {
  48.                     vtrs[i].dot(vtrs[i + 1]);
  49.                 }
  50.                 tmr.Stop();
  51.                 ed.WriteMessage("\nMy Dot Elapsed: {0}", tmr.ElapsedTicks);
  52.                
  53.                
  54.                 tmr.Start();
  55.                 for (int i = 0; i < points.Length - 1; i++)
  56.                 {
  57.                     points[i].GetVectorTo(points[i + 1]);
  58.                 }
  59.                 tmr.Stop();
  60.                 ed.WriteMessage("\nAc VectorTo Elapsed: {0}", tmr.ElapsedTicks);
  61.                 vtrs = vecs.ToArray();
  62.  
  63.                 tmr.Restart();
  64.                 for (int i = 0; i < vtrs.Length - 1; i++)
  65.                 {
  66.                     vtrs[i].CrossProduct(vtrs[i + 1]);
  67.                 }
  68.                 tmr.Stop();
  69.                 ed.WriteMessage("\nAc Cross Elapsed: {0}", tmr.ElapsedTicks);
  70.                 tmr.Restart();
  71.                 for (int i = 0; i < vtrs.Length - 1; i++)
  72.                 {
  73.                     vtrs[i].DotProduct(vtrs[i + 1]);
  74.                 }
  75.                 tmr.Stop();
  76.                 ed.WriteMessage("\nAc Dot Elapsed: {0}", tmr.ElapsedTicks);
  77.  
  78.             }
  79.  
  80.         }

gile

  • Gator
  • Posts: 2507
  • Marseille, France
Re: Can I get an assist from Math Geniuses
« Reply #30 on: April 18, 2013, 08:09:05 AM »
Hi,

Looking a little further at the results, I notice we were making a big mistake.
Compare the results of any previous command with one which runs sequentally...
None of the previously posted commands returns the expectyed result as foreach point in the collection, we're calling tree.NearestNeighbours(pt, 5) where pt can be any other point in the collection.

One way would be to get the nearest neighbours sequentially and compute the normals using parallel.

Code - C#: [Select]
  1.         [CommandMethod("GetNormals")]
  2.         public void GetNormals()
  3.         {
  4.             Document doc = Application.DocumentManager.MdiActiveDocument;
  5.             Editor ed = doc.Editor;
  6.             Database db = doc.Database;
  7.             RXClass rxc = RXClass.GetClass(typeof(DBPoint));
  8.  
  9.             using (BlockTableRecord btr = (BlockTableRecord)db.CurrentSpaceId.Open(OpenMode.ForWrite))
  10.             {
  11.                 ObjectId[] ids = btr.Cast<ObjectId>().Where(id => id.ObjectClass == rxc).ToArray();
  12.                 int len = ids.Length;
  13.                 HashSet<Point3d> pts = new HashSet<Point3d>();
  14.                 for (int i = 0; i < len; i++)
  15.                 {
  16.                     using (DBPoint pt = (DBPoint)ids[i].Open(OpenMode.ForRead))
  17.                     {
  18.                         pts.Add(pt.Position);
  19.                     }
  20.                 }
  21.                 Point3dTree tree = new Point3dTree(pts, false);
  22.                 Vector3d camera = ed.GetCurrentView().ViewDirection;
  23.  
  24.                 ParallelQuery<Tuple<Point3d, Vector3d>> normals = pts
  25.                     .Select(pt => new Tuple<Point3d, Point3dCollection>(pt, tree.NearestNeighbours(pt, 5)))
  26.                     .AsParallel()
  27.                     .Select(tuple => GetNormal(tuple, camera));
  28.  
  29.                 foreach (var item in normals)
  30.                 {
  31.                     using (Line line = new Line(item.Item1, item.Item1 + 200 * item.Item2))
  32.                     {
  33.                         btr.AppendEntity(line);
  34.                     }
  35.                 }
  36.             }
  37.         }
  38.  
  39.         private Tuple<Point3d, Vector3d> GetNormal(Tuple<Point3d, Point3dCollection> tuple, Vector3d camera)
  40.         {
  41.             Point3d pt = tuple.Item1;
  42.             Point3dCollection neighbors = tuple.Item2;
  43.             Point3d npt = Nearest(pt, neighbors), lpt;
  44.             Vector3d normal = new Vector3d(), tmp;
  45.             while (neighbors.Count > 1)
  46.             {
  47.                 lpt = npt;
  48.                 neighbors.Remove(npt);
  49.                 npt = Nearest(npt, neighbors);
  50.                 tmp = (lpt - pt).CrossProduct(npt - pt);
  51.                 if (normal.DotProduct(tmp) < 0)
  52.                     normal -= tmp;
  53.                 else
  54.                     normal += tmp;
  55.             }
  56.             if (normal.DotProduct(camera) < 0)
  57.             {
  58.                 normal = normal.Negate();
  59.             }
  60.             return new Tuple<Point3d, Vector3d>(pt, normal / normal.Length);
  61.         }
  62.  
  63.         private Point3d Nearest(Point3d pt, Point3dCollection pts)
  64.         {
  65.             double best = double.MaxValue;
  66.             Point3d ret = new Point3d();
  67.             foreach (Point3d pnt in pts)
  68.             {
  69.                 double cur = pt.DistanceTo(pnt);
  70.                 if (cur < best)
  71.                 {
  72.                     best = cur;
  73.                     ret = pnt;
  74.                 }
  75.             }
  76.             return ret;
  77.         }

« Last Edit: April 18, 2013, 10:21:34 AM by gile »
Speaking English as a French Frog

nobody

  • Swamp Rat
  • Posts: 861
  • .net stuff
Re: Can I get an assist from Math Geniuses
« Reply #31 on: April 18, 2013, 11:22:46 PM »
Can the code be updated to export the normal values (which I don't even know what that is)?  The software I am using is able to use the normal value like below:

740421.8800000000 3740249.9199999999 308.3200000000 -0.0078839810 -0.2789954587 0.9602600569

first three are coordinates and elevation, last three are supposed to represent a normal value (though I don't know what a normal value is)

Thanks!

WILL HATCH

  • Bull Frog
  • Posts: 450
Re: Can I get an assist from Math Geniuses
« Reply #32 on: April 19, 2013, 01:54:34 AM »
Hi,

Looking a little further at the results, I notice we were making a big mistake.
Compare the results of any previous command with one which runs sequentally...
None of the previously posted commands returns the expectyed result as foreach point in the collection, we're calling tree.NearestNeighbours(pt, 5) where pt can be any other point in the collection.

One way would be to get the nearest neighbours sequentially and compute the normals using parallel.
I'm not quite sure what you mean here, but this version has serious performance losses compared to your last version, I'll try to make sense of why over the weekend.

WILL HATCH

  • Bull Frog
  • Posts: 450
Re: Can I get an assist from Math Geniuses
« Reply #33 on: April 19, 2013, 02:25:09 AM »
Can the code be updated to export the normal values (which I don't even know what that is)?  The software I am using is able to use the normal value like below:

740421.8800000000 3740249.9199999999 308.3200000000 -0.0078839810 -0.2789954587 0.9602600569

first three are coordinates and elevation, last three are supposed to represent a normal value (though I don't know what a normal value is)

Thanks!

the normal is perpendicular to the surface

replace the portion where we draw the lines with the following:
Code - C#: [Select]
  1.                 using (StreamWriter sw = new StreamWriter(new FileStream(db.Filename.Substring(0, db.Filename.Length - 4) + ".txt" ,FileMode.Create)))
  2.                 {
  3.  
  4.                     foreach (var item in normals)
  5.                     {
  6.                         double[] output = item.Item1.ToArray().Concat(item.Item2.ToArray()).ToArray();
  7.                         sw.WriteLine(string.Format("{0} {1} {2} {3} {4} {5}", output[0], output[1], output[2], output[3], output[4], output[5]));
  8.                     }
  9.                 }

gile

  • Gator
  • Posts: 2507
  • Marseille, France
Re: Can I get an assist from Math Geniuses
« Reply #34 on: April 19, 2013, 02:49:23 AM »
Quote
I'm not quite sure what you mean here, but this version has serious performance losses compared to your last version, I'll try to make sense of why over the weekend.
Sorry my english isn't so good to give a clear explanation.
If you try to run my last "all parallel version" or one of yours two (or more) times on the same points coud, you'll see the results are different each time.
This is due to the calling of tree.NearestNeighbours(pt, 5) in a parallel execution. There may be as different pt as the number of cores processed at the same time and the pt returned as Item1 of the tuple may be different from the one used to compute the vector in the tuple Item2.

Before improving speed performances we have to make sure the method returns the expected results. IOW, start with a strong sequential method and then try to parallelize what can be without returning wrong results.
Speaking English as a French Frog

gile

  • Gator
  • Posts: 2507
  • Marseille, France
Re: Can I get an assist from Math Geniuses
« Reply #35 on: April 19, 2013, 11:36:49 AM »
OK, I got it.
This issue was due to the using of (global) fields in the Point3dTree class.
I updated the code here.
The parallel execution seems to work fine now.
Speaking English as a French Frog

nobody

  • Swamp Rat
  • Posts: 861
  • .net stuff
Re: Can I get an assist from Math Geniuses
« Reply #36 on: April 22, 2013, 10:35:52 PM »
I can't thank you enough gentleman! Your code produced exactly what I needed, which enabled me to produce the model I was looking for: