Behavioral object based tracking

(transcribed from copy of video which came out too big.. 2GB)

Hello, good morning. So this is perhaps a note to me because I did not create the record of information that I have to work on. So I was looking for jobs and I of course was just graduated from doctoral.

I had worked recently in object tracking using, well using whatever there is. And I did work on tracking using Monte Carlo particle, Rao-Blackwellized based version of the tracker which used the statistical approach, tracking and obtained the result and the particles somehow with the livelihood give its robustness that it can get more better. Although now the point was that each object is different and each object has a different trajectory so the goal is to model the object’s behavior and I guess this might have been done before.

Well, I only read one paper which was motion GPT but it was a complex human motion which is a body which has multiple kinds of motion for one body while currently I will start with a single simple object which is just a point object and it only has, the object exists and it exists in a space and can only occupy one position at a time and only has one velocity vector and one position vector and so it’s not an object made of two different entities with different velocities, it’s just one point and the goal is for an article to make the, to make the, to show that we have this some model which is basically would be some form of machine learning model although the start is just going to be the model for the like mathematical model of different trajectories and of course that is captured by the EKF for nonlinear systems for Kalman filtering but that’s not the point I think that in a normal state-space model by nonlinear models and models we are just trying to model that f there which is already existed it’s just nothing new I think the publication would only come from the value by ML or the data trainable system although if that is desired in some sense because we don’t want to put hands on brains when you want to make systems computers or robots predict the trajectories they should be done on data and although the prior and by intuition and thinking of course it is possible to construct one model and among all kinds of models well I have made some notes so in the volume of cubes we will be talking about the models for the models for this three-dimensional space that exists and surrounds us that we feel and that would be for an object in that space which is a point object and so the goal would be to go the mode point can move so in that volume of cube it’s the point is at the center and it can it has this it can move in the future trajectory and if the object has some identity it has some behavior to it that can be also put as a prior to plot to restrict the prediction or motion prediction of where the object can be if the doesn’t have any identity it’s just uniformly randomly it could be anywhere it could move anywhere or just stay there depends if the object is a static object it will stay there and object if it’s a dynamic of course is a Brownian motion that we use for molecules that’s one direction of this but we are looking for some precise or flowing trajectories with particular flow that will be to the object or many objects actually and but for now we will be considering a single object and the object did not need to be a point it can be extrapolated to a Gaussian although that would mean that could mean more complexity of things that could add not not that difficult though but we will be still for mathematical simplicity we’ll just use point and so I started drawing some trajectories of how well the object can move if it had a behavior of course it could be random but we are the the in the real life the birds move in certain sense the humans move in certain sense if you look at them at points of course they are complex bodies with complex motion for each point and that also defines effects and but we will be looking at these objects in the behavior in the nature as point objects for example a car you just move in a straight line usually unless there is a turn and you know why the turn happens is because of the structure of the car is taking place the aeroplane they typically go straight lines and the large curves are or a helicopter and how these objects behave what else is there maybe a person doing a skateboard or or vice person bicycling or person walking running a machine doing excavation job or just moving around stuff digging construction area how the motion is restricted and so well it could be very easy to visualize but for always you could have this space of motion in which the object could move and the space can be looked as in near future how much the object can move so for example a car going a straight line it’ll always be a straight line with some window around it that we can move into and you could associate from all those set of pathways some probability to it although that’s not much useful at the moment because you just need to know if that there is a trajectory even because the probable path cannot be likely because of external disturbance which is unknown so it’s kind of very the randomness is actually random basically getting sampling from for one direction which is improbable can get affected by some other random outcome that you get an improbable outcome which was probable and because of the randomness in this that you cannot choose and so yes so so we will be only looking at like discrete like very fixed trajectories because so far now we will not be assigning probabilities although that is what we would be popular to do actually in the Bayesian sense but we will be only looking at simple classical way and so the amount of trajectories a point can have in a cube if that’s the center it can go it’s infinite but we look at the point’s motion not as through time but as from the 4d space we take that trajectory as an entity or object or what it is it is the object points future with the present and that entire thing is called trajectory so we are dealing in 4d space because we have we are in working with time and so object is a it’s a 4d object a trajectory and for this point and it’s object i mean the how of the how you look at it basically wait a moment okay so there are infinite possibilities of the trajectory of these 4d objects and so for particular behavior of objects we expect object to go a certain way and then that that we would sample this entity of trajectory a 4d object from this all space of possibilities for a given object and yes and another thing is object exists there’s existence of object when object is brought into space is born it’s the birth of an object and when object ceases to exist it’s it’s the death of the object although we could use a different case where the when object vanishes it’s called it the object doesn’t really die but actually is you can say it’s in some another plane or space and then then it can appear from another place and that would be very close to what people would see the life as but or want to believe that but of course that is another unknown thing you can consider the object when dies it ceases to exist but in real world of course the things can produce and annihilate but for the things such as card cardboard and like most of them they are not of course dying or vanishing they are existing they are not going into a wormhole or teleporting to someplace else yeah if you had those things then you would have a different a bit different model that you need to take into account but of course for normal life the object exists and it continues to exist in the space if it moves to a volume that that it is constrained to it will be in that volume it is not going to not go anywhere else although it can be if it was some portal then of course you would not find it there in that space so well watch out for wormholes anyway so so for the object to have a future trajectory is a simple delta motion is this the small amount of motion can be captured by just using three numbers and that is delta x delta y delta z or in polar coordinates theta 5 delta r or you could just use a displacement vector and these were three dimensions but of course you could do any dimension but for three dimensional space yeah three numbers three numbers of that precision in the computer that are required and we will use discretization because in these all systems are we are based on the concept of numbers and on a computer which uses number representation through binary and of course we are abstracting all that to for scientific purpose so we will discretize to a certain delta t time step and so the goal is for that point to create a model behavior that predicts the next n steps of that point and then the prior information can be all the history of the point or past m samples are used to produce next m samples or or it could be just a fast sample to produce a nice sample and and and a simple version for this delta motion you just add to the current space mathematically just add the displacement to get the new object and that is our first model it’s not a machine learning model but it is what machine learning is it’s not a very high level it’s just very low but it works perfectly for perfectly straight motion and this is the only model for that if you want to track objects in a particular setup direction just give the angle and then you know comparing the past and the future given the angle you know which object is which and well of course you could track stars in an image with it for example stars only might appear to move in a straight line in the small area of the image so you could only always use this linear model to match and let’s say correlate the image across that angle to get the displacement of how much they have appeared to move or how much the earth has rotated but that is the simplest model that is actually works you don’t need to do anything more but the local world that we see is not plain motion for our time of observation that we have they take complex motion and for that we need to go to higher kind of borders and so first with delta step that was 1 but now we could have 2 or 3 or n such deltas and then again we have those n such deltas we can we just add them to the position to get like x plus delta 1 x plus delta 1 plus delta 2 x plus delta 1 plus delta 2 plus delta 3 and you can just have the three trajectories of the uh have the single trajectory the three next three points for example or we could just compute all of them at once just by computing delta 1 delta 2 delta 3 summing them up net displacement after three time steps net displacement after two time steps and next displacement so yes and or we can go sequentially we can compute the next one first then produce the next one and the next one and having more deltas would just give finer detailed model of how the behavior is and it can have complex behavior because since there are only three numbers for delta 1 for producing two deltas you have six numbers so the motion is a bit more complex but we are in in reality we want to go till deltas of maybe 10 i guess if we have a sufficient delta step sufficient discretizations then then we will have this complex enough model for our use case of tracking different complex objects and so so that is just by adding those three points instead of this we just call them a trajectory the displacement trajectory up on the point so if it was just a point you could have a displacement trajectory which is of course all we are assuming is a deterministic system there’s no noise yet and the past whatever we define it is the truth it is the correct answer to it so what if you say the object moves by one that means you assume and know that moves by one one step our trajectory and how it is so so from the point we only add that trajectory to it and that’s the model in the we could use straight up the transformer architecture at some point because it uses attention to select what class it is and then then it will apply that machine learning model using resonant style architecture basically it computes x and r saying the result of the network to itself and that stage itself is the thing that because we don’t need to compute the entire trajectory given the trajectory if it could affect of course we are looking at the sensor local space given x how to produce the output x plus fx gives the next output in this network architecture but so in the displacement we just add that fx as the displacement trajectory and the machine learning model would just know the displacement trajectory given the input or inputs of the past history that it has and and we could just try to incorporate in that kind of way and there’s another aspect to it and neural networks they feed forward network when adding it also normalizes it’s basically scaling down when you add something to something and then you just scale down to your viewpoint and that the viewpoint is only useful for the next observer in the layer in the network basically which takes that as the input and and then processes it so you can see the network as this is small computing units which are they are but you can see them network not as a single unit but it’s a smaller detailed units much more like mister the final trajectory to just be moves here but the details are given by individual layer networks and a complex enough motion model if it’s a single feed forward layer network it can only have one kind of simplicity like say two or three steps or just just a linear model but consecutively of course they could add although not exactly you have to see how non-linear can a feed forward neural network you could put there and accordingly it would be variable and so that’s from network point of view and well it’s basically blurring the boundary of course it’s no no such discrete discrete separate thing and science is the same thing the life has been the same ancestors thousand years ago knew how to live and we also know how to live we just live at the detailed level grain level of belief and and the surroundings we are just opened up more of what there is the sciences digs deeper the connections stay the same if they are facts and verified of course then the truth remains the truth you just broke the truth down into pieces which are the pieces of this truth that we study using science and makes a deeper connection and that’s what new thing is for now and i think that’s all i could come up for now so i decided to make a video because it’s already 22 minutes i think it’s a bit too long but anyways i will stop here