Celestial correction for Great Lakes?

[Bigjim]

I was a navigator in the Navy and learned celestial navigation. I just taught my second class here on Lake Michigan. A good celestial LOP will be less than 10 miles from your actual position.

During our class we got sights on the sun, moon and Mars. Oddly, all of our lines landed within 5-7 miles of the class location but always to the north.

So, I’ve always wondered but can’t find any information on whether there is a special correction for the Great Lakes. We’re 570 feet above sea level. But our sighting are based on the horizon of Lake Michigan, not the ocean.

Airplane navigators apply altitude correction but they are not using the horizon in most cases.

Does anyone know for certain?

[billknny]

What you are doing is making your measurements on a virtual planet that is effectively 1140 feet larger in diameter (2 x 570 feet) than the "real" earth. Not the same correction as a "height of eye" because as you correctly point out your horizon is higher too, but a correction all the same.

I have not seen such a correction tabulated anywhere, but it is in the direction you are observing. An error always due north for a body near the ecliptic and near your meridian, with additional error to the west for a body to the east of your meridian, and to the east for a body to the west of your meridian.

Draw out the angles, you'll see the issue.

[SeanPatrick]

Originally Posted by Bigjim

Oddly, all of our lines landed within 5-7 miles of the class location but always to the north.

Interesting. I'd love to help, but I'll need some more information first:

1. Are you taking sights from the shore, or out on the lake? IOW, are all of your sights in roughly the same direction (if so, which direction), or are they spread out through 360°?
2. Is your horizon at least 5 nautical miles away? Is there a possibility that there is another shore or some other obstruction at or near the horizon which is less than 5 NM away in the direction of your sights?
3. What method are you using to reduce the sights? (e.g. Pub 229, 249, a calculator or computer/phone app.)
4. Are you taking parallax in altitude of the Moon and Mars into account?
5. Are you using the same sextant or multiple sextants for these sights? Have all possible instrument errors been accounted for or adjusted out?
6. Have you checked your timepiece for error? How, specifically, are you getting the times of each sight?
7. Are you taking temperature and pressure into account? (Certain atmospheric conditions can have a noticeable effect on sextant measurements.)
8. Most importantly: can you post the data from one or more of your erroneous sights, including as much information as possible so that I can try and work through them myself? (Please also include the actual lat./lon. from which the sight was taken.)

To be honest (and with all due respect), I'm not really sure what billknny is trying to say. I have never heard of the type of correction he mentions - in any navigation text. If you are using a natural horizon that is far enough away and you have correctly adjusted for height of eye (and other standard corrections), then there is really no reason for the error you describe - no matter what your elevation above sea level is. I suspect that is why he has also never seen such a correction tabulated anywhere.


To be sure: there is no "special correction" for the Great Lakes - or any other location on Earth. All of the standard corrections will work for any location, as long as they are applied correctly.

[Adelie]

Try posting your question to NavList. They’re the heavy hitters for CN questions these days.

[SeanPatrick]

That's a good idea. I'm a member of NavList, btw ... but more minds on the problem can lead to the answers faster.

[El Pinguino]

Originally Posted by SeanPatrick

Interesting. I'd love to help, but I'll need some more information first:


[LIST=1]
[*]Are you taking temperature and pressure into account? (Certain atmospheric conditions can have a noticeable effect on sextant measurements.)
.

Abnormal refraction... could be an issue in the summer on the Great Lakes I guess...

Special tables for it in the front of both the almanac and - I think - the air tables...

[SeanPatrick]

Yes. You can also use the following formula:

(0.28•P)/(T+273)

... where "P" is the pressure in mb and "T" is the temperature in °C.

Multiply the refraction correction by the result.

[inHg•33.86=mb; (°F-32)•(5/9)=°C]

[Bigjim]

Originally Posted by SeanPatrick

Interesting. I'd love to help, but I'll need some more information first:

1. Are you taking sights from the shore, or out on the lake? IOW, are all of your sights in roughly the same direction (if so, which direction), or are they spread out through 360°?
2. Is your horizon at least 5 nautical miles away? Is there a possibility that there is another shore or some other obstruction at or near the horizon which is less than 5 NM away in the direction of your sights?
3. What method are you using to reduce the sights? (e.g. Pub 229, 249, a calculator or computer/phone app.)
4. Are you taking parallax in altitude of the Moon and Mars into account?
5. Are you using the same sextant or multiple sextants for these sights? Have all possible instrument errors been accounted for or adjusted out?
6. Have you checked your timepiece for error? How, specifically, are you getting the times of each sight?
7. Are you taking temperature and pressure into account? (Certain atmospheric conditions can have a noticeable effect on sextant measurements.)
8. Most importantly: can you post the data from one or more of your erroneous sights, including as much information as possible so that I can try and work through them myself? (Please also include the actual lat./lon. from which the sight was taken.)

To be honest (and with all due respect), I'm not really sure what billknny is trying to say. I have never heard of the type of correction he mentions - in any navigation text. If you are using a natural horizon that is far enough away and you have correctly adjusted for height of eye (and other standard corrections), then there is really no reason for the error you describe - no matter what your elevation above sea level is. I suspect that is why he has also never seen such a correction tabulated anywhere.


To be sure: there is no "special correction" for the Great Lakes - or any other location on Earth. All of the standard corrections will work for any location, as long as they are applied correctly.

All good questions.

1 We took all sights from the deck of the Yacht club with a height of eye about 12 feet above the water. All sights were corrected for DIP.

2 Since the club is on shore, we can only shoot the sun or other bodies in the morning facing east and south east. Lake Michigan is at least 80 miles at this location. Other sightings we take use Davis artificial horizons.

3 The method I learned in the Navy uses the Nautical Almanac and HO 249.

4 We applied ALL corrections requires for all sightings and bodies. That includes refraction, dip, d and v (as necessary), time, index, temperature and barometric corrections. The point of my post is that ALL our LOPs landed the same distance NORTH of our actual location.

5 In our class, we had 3 Davis plastic sextant and three metal Japanese models. All were checked for index errors and corrected after each sighting. We worked LOPs off of each sextant.

6 Time was based on Time.gov. This sight automatically corrects for network lag.

7 As noted in #4 we applied all corrections. Please note that at 42 degrees north with altitudes of 40 degrees at the extant temperatures these corrections would be tiny or zero. Yes, we checked barometric pressure. In all instances with or without these corrections the LOPs were all north of our actual position.

8 I will post some of our sights in a separate comment.

To me, it makes sense that the height above sea level would affect the outcome. All reduction tables are based on sea level which is over 500 lower than our position. Remember, our sightings are based on the horizon of Lake Michigan not the ocean. As noted in another reply, this would create angles for larger planetary circumference. How could it not affect the result?

[El Pinguino]

Originally Posted by Bigjim

we can only shoot the sun or other bodies in the morning facing east and south east. The point of my post is that ALL our LOPs landed the same distance NORTH of our actual location.

A position line obtained from a body bearing to the east of you can not 'land' to the north of you as the position line runs north/south..... you can be anywhere on that position line... it only gives you longitude


6 Time was based on Time.gov. This sight automatically corrects for network lag.

An error in time would result in an error in longitude not latitude





To me, it makes sense that the height above sea level would affect the outcome. All reduction tables are based on sea level which is over 500 lower than our position. Remember, our sightings are based on the horizon of Lake Michigan not the ocean. As noted in another reply, this would create angles for larger planetary circumference. How could it not affect the result?

HO 249 are 'Air Navigation Tables' https://www.celestaire.com/product-c...ir-navigation/ designed originally for use in aircraft.... they are far more than 500 feet above sea level... being 500 feet above the level of the ocean rather than your lake puts you 1/10th of a mile closer to a body 8 light minutes away.... neither here nor there methinks

Other sightings we take use Davis artificial horizons.

This is where I suspect the problem lies as these are the sights where you would be getting a P/L lying E/W that will give you an error in latitude.......

I've never used an artificial horizon ... however I do know that some of the corrections are applied in the opposite direction or are doubled or somesuch....

I think you will find something about corrections using artificial horizons in my scribblings down below... https://www.dropbox.com/s/a5blh1rgvi...ation.pdf?dl=0


The fact that the error is constant suggests the same mistake in application of the corrections is being made all the time and by all the people.

At least its not a random error.......

I suggest getting a boat.... going far enough offshore one day to get a clear sea (lake?) horizon to the south and taking some sights while out there....

[Bigjim]

Originally Posted by El Pinguino

Other sightings we take use Davis artificial horizons.

This is where I suspect the problem lies as these are the sights where you would be getting a P/L lying E/W that will give you an error in latitude.......

I've never used an artificial horizon ... however I do know that some of the corrections are applied in the opposite direction or are doubled or somesuch....


I suggest getting a boat.... going far enough offshore one day to get a clear sea (lake?) horizon to the south and taking some sights while out there....

Using the artificial horizon you simply divide the reading by 2. There is no DIP correction.

All sightings using the actual horizon were taken BEFORE noon. So, the azimuth of the sun is toward the southeast. No shots were made directly to the east.

If you are aware of what an LOP is, it is NOT a fix. It is simply a portion of an arc. The result of the sighting provides a circle of "equal altitude". The circle can be several thousand miles in diameter, centered on the ground position of the body.

When you take a sighting several hours later you are creating a second circle. The two circles should intersect at your location. But, there are TWO intersections, one north and one south. The observer can be at either intersection. That's why you need to know your assumed latitude and longitude.

All the LOPs we plotted were north of our actual location.

Here is a screen shot of our typical plots combining LOPs from different times:



The horizon does NOT have to be to the south. It has to be below the body. In the morning you will be looking toward the east or southeast. At noon, the bearing will be toward the south. In the afternoon and before sunset, the bearing will be toward the southwest and west.

As for aircraft, my friend was a navigator in the air force. He shot sightings using a bubble sextant. He would shoot at various times of the day but did not use the actual horizon. In most cases, he was well above 500 feet. But the altitude should not matter as long as you correct for it.

[El Pinguino]

Many thanks for the lesson on celestial navigation...........

The two sights taken before noon each appear to have an intercept of about 8' 'away'.... which is how it would be expressed if you had used Marc St Hilaire ( longhand) rather than the air tables.
The afternoon sight taken with - I assume - the artificial horizon gives an intercept of double that... about 16' 'away'....I'm having to just eyeball this from the thumbnail....

This suggests a number of things to me... firstly refraction either applied the wrong way or 'abnormal'. Assuming altitude greater than 20* additional corrections for 'standard' low altitude refraction should not have been required....

The fact that the intercept has doubled when using the artificial horizon suggests the original error has been compounded .....

I am assuming that multiple sights have taken by multiple people using multiple sextants which would rule out any sort of personal error....

[Bigjim]

First, you can click on the thumbnail to see the full-sized image.

Second, those were approximations of the actual plots. But close enough.

Third, the general observation is still pertinent. All LOPs were north or farther. I've never seen a pattern like this.

When I was in the service, my LOPs fluctuated short and long.

I'm not concerned that one might be 6' and the other 10'. All of them were within tolerances.

And, yes, multiple people took sightings but in most cases, we used my own since I was teaching the class and had the best experience with sextants.

[Benz]

Another vote for personal error. It's my understanding (insofar as I understand this at all...) that the tables calculate using the zenith distance--that is to say, the distance between the object and the zenith of the observer, rather than the distance above the horizon as seen by the observer, though that's actually what the observer, well, observes. How the input is converted without my converting it is still beyond me, but I remember a handy diagram in Bowditch that explains it pithily. Which explains why you can use a bubble sextant or regular sextant to get the same results, though the bubble is, of course, oriented toward the zenith.

[SeanPatrick]

TL;DR: I still need your raw data in order to make an educated guess as to what went wrong here.

Originally Posted by Bigjim

8 I will post some of our sights in a separate comment.

I eagerly await your data. It is the only way to figure out if this might be a mistake somewhere in the reduction process or some other type of error which has escaped your attention. A most interesting mystery.

Originally Posted by Bigjim

To me, it makes sense that the height above sea level would affect the outcome. All reduction tables are based on sea level which is over 500 lower than our position. Remember, our sightings are based on the horizon of Lake Michigan not the ocean. As noted in another reply, this would create angles for larger planetary circumference. How could it not affect the result?

Consider this: the largest correction involved in sight reduction for the observer near the surface of the Earth is parallax in altitude of the Moon. This correction is required because one is comparing an observation made on the surface of the Earth [topocentric] to a position calculated as if the observer was at the center of the Earth [geocentric]. (Sight reduction tables are based on geocentric positions, not sea level.) Because the Moon is much closer to Earth than any other celestial body, this correction can exceed one whole degree. For Venus and Mars the correction is usually between 0 and 0.2 arc minutes. For all other bodies, parallax is negligible.

Now, think about the difference between these two positions (topocentric and geocentric). It is (on average) 6,371 kilometers - or about twenty one million feet. That means that the difference between an observation made at sea level versus an observation made 570 feet above sea level would be about 0.003% of the correction needed for a low altitude Moon observation - around 0.1 arc seconds. This is way beyond the abilities of even the most skilled observer using a marine sextant.

But all of this is really a moot point because, as I explained above, we are not comparing to an observation made at sea level. Furthermore, we are not even comparing observations based on the true horizon, either. The correction for "height of eye" (dip) takes care of that. And the parallax correction takes care of the difference between our topocentric observation and the geocentric calculation. IOW, all of these potential differences have already been accounted for.

I considered mentioning the case of celestial navigation in airplanes (as El Pinguino mentioned) in a previous post - but chose not to. In addition to what I have already explained, this decision was based on the fact that air navigators do not use the true horizon, but instead use an artificial horizon (usually in the form of a bubble - essentially a level) which does not require a dip correction. Again, the difference between an observation made at sea level and and one made at 30,000 feet is really inconsequential when it comes to celestial navigation.

[Bigjim]

Originally Posted by SeanPatrick

TL;DR: I still need your raw data in order to make an educated guess as to what went wrong here.



I eagerly await your data. It is the only way to figure out if this might be a mistake somewhere in the reduction process or some other type of error which has escaped your attention. A most interesting mystery.



Consider this: the largest correction involved in sight reduction for the observer near the surface of the Earth is parallax in altitude of the Moon. This correction is required because one is comparing an observation made on the surface of the Earth [topocentric] to a position calculated as if the observer was at the center of the Earth [geocentric]. (Sight reduction tables are based on geocentric positions, not sea level.) Because the Moon is much closer to Earth than any other celestial body, this correction can exceed one whole degree. For Venus and Mars the correction is usually between 0 and 0.2 arc minutes. For all other bodies, parallax is negligible.

Now, think about the difference between these two positions (topocentric and geocentric). It is (on average) 6,371 kilometers - or about twenty one million feet. That means that the difference between an observation made at sea level versus an observation made 570 feet above sea level would be about 0.003% of the correction needed for a low altitude Moon observation - around 0.1 arc seconds. This is way beyond the abilities of even the most skilled observer using a marine sextant.

But all of this is really a moot point because, as I explained above, we are not comparing to an observation made at sea level. Furthermore, we are not even comparing observations based on the true horizon, either. The correction for "height of eye" (dip) takes care of that. And the parallax correction takes care of the difference between our topocentric observation and the geocentric calculation. IOW, all of these potential differences have already been accounted for.

I considered mentioning the case of celestial navigation in airplanes (as El Pinguino mentioned) in a previous post - but chose not to. In addition to what I have already explained, this decision was based on the fact that air navigators do not use the true horizon, but instead use an artificial horizon (usually in the form of a bubble - essentially a level) which does not require a dip correction. Again, the difference between an observation made at sea level and and one made at 30,000 feet is really inconsequential when it comes to celestial navigation.

You really sound like you know what you are talking about. But there's one thing you are ignoring. The DIP correction for an altitude of 577 feet is 24' according to the Air Almanac. So, 500' may be insignificant to the distances of the celestial bodies but obviously it's not insignificant to the calculations.

A correction of -24' is HUGE. But, this assumes the reading is off the "actual" horizon. As you noted, aircraft usually use a bubble sextant. We are using the horizon of Lake Michigan.

So, I wanted to perform a simple proof of your contention. Below are two diagrams. Assuming an observer could be at the exact same position but take readings from two different altitudes.


(Click on Thumbnails to see full size image)

The first diagram illustrates YOUR contention that altitude should make no significant difference in the readings. (YES, my diagram is greatly exaggerated.) Since the Sun in 93 million miles away, the difference in the angles would be very slight. But, as shown, the identical angle does NOT resolve to the same position on the sun.

The second diagram, shows an adjustment to the reading to match the center of the sun. So, in fact, altitude can have a impact on the observed angle.

You seem to suggest that 500 feet of altitude is insignificant while the DIP table contradicts your supposition.


(Click on Thumbnails to see full size image)

DIP corrections apply dramatic changes to the sextant reading. Our readings were taken just 12 feet above the water level. The DIP correction is -3.4'.

Our LOPs were between 6 and 10 miles north or away of our actual location. There are lots of reasons these intercepts could be off. But we applied all known corrections. You seem to think the differences are all human error or other factors. My contention is that there is ALWAYS human error, but the fact that all intercepts landed away from our actual position speaks to something else.

Statistically speaking, some of our sightings should have fallen short of our position if it was simply human error. To me, the difference between 6-10 miles is the human error. The rest may point to an additional correction for "altitude of sea level".