This is a little complicated to explain but this is my best attempt:

im converting a raster image into a vector. Its a simple image that has shapes and hard lines. It has a grid overlaying it. Lets say 100px at each grid line. Ive decided to make the vector image at 10mm/grid line. This comes out to a dpi of 254. 

Lets say the whole image is only 300px X 300px. This means the vector document is 30mm X 30mm. Whenever is match the raster image to the vector canvas, on the top bar it is NOT 300px. It is 30mm but it cannot be correct in both scales, only 1 at a time.

The frustrating part is the actual document is more complicated and has multiple different images that are multiple different "scales". So one image will be 50px/10mm and another 100px/mm and some scale perfectly if i set their DPIs in the object properties and others completely ignore it and use the default 96. I need them to be correct in both because the images are not in nice even numbers and not all of them have this grid to easily line up. I can manually line up most but the properties dimension values are all wrong. This means when exporting, the overall canvas size, at the correct DPI & Correct mm scale, exports an incorrect image resolution. I have changed all of the preferences options that mention dpi from 96 to the correct one, ive changed the document properties, ive changed all the individual object properties and still theyre all incorrectly sized. Visually they can all line up, but where it should for example say "100px" in height, itll say "241.493px" i cannot work with these values for scaling or changing because the math wont add up

Is there any way to adjust this or is it just something im gonna have to work around by making it look visually correct on the canvas and deal with exports by scaling the entire image to a custom export resolution. Which i imagine will stretch and warp the image. 

If the goal is converting raster to vector, and the design is composed of shapes and lines, why the concern of dpi?  Vector objects don't have pixels...

My recommendation is if you are working with images where DPI matters set the display units to mm, inches or your other favorite physical unit and work with that. Ignore the px dimensions displayed in most places of Inkscape UI. Those only make sense if you are working with px only drawings something like a website banner, icon or a game assets. They have no direct relationship to px size of your imported raster image.

If it doesn't make sense to work in physical units it also doesn't make sense to talk about DPI. DPI is relationship between physical size and pixels. Without physical size (inches) then there is dpI (dots per inch). If you have only px, then you have "pixels per pixel" which is just a scaling factor.

> Lets say the whole image is only 300px X 300px. This means the vector document is 30mm X 30mm. Whenever is match the raster image to the vector canvas, on the top bar it is NOT 300px. It is 30mm but it cannot be correct in both scales, only 1 at a time.

If you want your 300px X 300px bitmap image to be 10px/mm set the image dimensions to 30mm and that's it.   The px you see in top bar or ruler if you switch display units to px doesn't matter, it doesn't have to show 300. Just because you see a number which differs from 300px there doesn't mean Inkscape has resampled your bitmap to 241.493px. It hasn't chopped of half a pixel. The scaling process of imported bitmaps in Inkscape is lossless. If the resolution of your bitmap is 300px that's what is, whether you can see all of those 300px depends on your zoom level. And it will remain so even after exporting as long as you export to a vector art format like SVG or PDF. Only if you export your svg as raster image the image will be rasterized and scaling losses might apply. 

>  if i set their DPIs in the object properties and others completely ignore it
The DPI in object properties is only for embedded SVG files. It doesn't matter for your imported raster images.

For the imported raster images Inkscape will either use DPI embedded in raster image metadata or use the default value from settings. Depending on how you have configured Inkscape it will ask during import or pick automatically. Note that this is only affects initial size after import. If you decided to calculate dimensions manually and resize it, then import DPI doesn't matter. Remember that the amount of pixels in imported raster is whatever software that created outputted, and the physical size is whatever dimensions you enter in top bar (preferably using physical unit unless you want to confuse yourself).

> his means when exporting, the overall canvas size, at the correct DPI & Correct mm scale, exports an incorrect image resolution.

If you did everything correctly and worked in physical units during export choose the size if image that corresponds to it's area, choose the DPI you want exported image to be and export. The resolution will be calculated based on physical size and DPI. That's how DPI works. You can't freely change all 3 at the same time. Iit doesn't how to match with the DPI of imported image.  You can set the same DPI as imported images in case they had appropriate scale and you want to maintain pixel accurate result. But that only works if you haven't done additional resizing and all imported images have same DPI. You said it yourself that you have a bunch of different images at different scales in such case it's unavoidable to have mismatch between export pixels and at least some imported raster pixels since it can't be multiple different values at the same time.

Export DPI only matters if you are exporting as raster image. If you are exporting as SVG or PDF it doesn't matter. (*there is an option for DPI of rasterizing effects but that only applies to inkscape specific effects/filters which can't be directly mapped to the primitives provided by PDF, and is completely different from other DPIs discussed this)

(Yes I know technically in all the places I mentioned DPI it is more accurate to call it PPI pixels per inch. The dots created by various printers are neither square nor aligned in neat grid like pixels.).

One exception to the recommendation above about working purely with physical units is when the DPI ratios between all your images is whole integers you want to keep everything pixel aligned (For example, 100DPI, 200DPI and 400DPI). In that case there are two options: work in pixels only using either highest or lowest DPI as reference (ignore physical units), or work in physical units but setup grid based on common DPI of your raster images for aligning them.  Again this only makes sense if DPI make nice ratio, if you are dealing with  a random mixed  DPI images for example 96, 125, 150 and 240, it will not work they will be misaligned no matter what.