Target identification fusion based on multi-source remote sensing images can make full use of the redundancy and complementary information from all sensors

acquiring more accurate result of target recognition.One of the pre-condition of identification fusion is target association

which is to determine if the information from two or more images are related to the same target and should be fused together.Due to different performance of sensor and diverse target distribution

the extracted information of targets generally has some uncertainty

which results in the difficulty in judging whether the information from two images is originated from the same target.Therefore

how to utilize the information of remote sensing images to distinguish multi-target association has become an urgen problem.There are two kinds of methods concerning target association when using image data: one is Kalman filtering based data association and tracking

which utilizes accumulated kinematic information of multi-frame images to estimate and track.Typical methods are Nearest Neighbor（NN）

Joint Probabilistic Data Association（JPDA）

Multiple-Hypothesis Tracking（MHT） and so on.These methods need dense sampling of observed data

and the target motion model should be simple.The other one uses image match in computer vision for reference.Typical methods are cross correlation matching

feature matching and so on.These methods usually work on condition that only single target is concerned.For remote sensing images

there are two problems when associating multiple targets in them.Firstly

it is incapable to acquire a series of multi-temporal remote sensing images on the same region at present

so the kinematic state of a target cannot be estimated accurately with low temporal resolution data and the classical Kalman filtering association algorithms are no more applicable.We must seek for other time-independent information as the associating measurement

which can be image invariant feature.Secondly

there are two uncertainties lying in image feature extraction of a target.One uncertainty lies in determining invariant features due to various image distortions such as rotation

scaling and so on.The other lies in establishing feature correspondences between any two consecutive images.So

it is difficult to discriminate the ambiguity of multiple targets’ correspondences when using image matching-based association method.In order to solve above problems

a novel multiple targets association method based on image invariant feature matching and Association Cost Matrix（ACM） global optimization is proposed.At first

the Multi-scale Autoconvolution（MSA） transform of a target is computed based on affine invariant theory and is used as association measurement

which can overcome the negative influence of changes in target’s pose

imaging viewpoint and so on.Secondly

the association cost matrix is constructed based on the dissimilarities of MSA feature matching of any two target pairs from two images respectively

representing the correspondence illegibility of two targets.Finally

the minimal energy of ACM is found using simulated annealing（SA） algorithm

and the global optimal association result is achieved.From the simulation experiments

some conclusions can be drawn as follows:（1） Using image invariant features to perform target association is a validate way

overcoming the bottleneck that the time-dependent kinematic feature cannot be estimated from sparse remote sensing image series.（2） Compared with the NN local algorithm

the optimization of association cost matrix is a global optimal algorithm and has excellent performance in dense targets circumstance.（3）The approximate algorithms such as SA can greatly improve the search of optimal association cost matrix

and then make complex association method practicable.