I've not seen anyone record
first use in written English
of 'tell-tale' associated with a sail.
Term was used for mechanical contrivances, such as a pointer placed near wheel
that showed rudder
Another term, dog vane, is recorded from 1769. A dog-vane, was a tell-tale 'formed of a piece of pack-thread about two feet in length' attached to the gunwale, to show direction of wind
uses 'tell tale' in 1838 in _Homeward bound_, but probably for a dog-vane, not a leech tell tale. Melville uses 'tell tale' in Moby Dick, but not for a sail tell tale.
Feel free to dig through https://books.google.com/ngrams/graph?content=tell+tale&year_start=1800&year_end=2 000&corpus=15&smoothing=3&share=&direct_url=t1%3B% 2Ctell%20tale%3B%2Cc0
First use of tell tale for trimming a fore-and-aft sail should be somewhere there.
Yachting magazine of 1964 definitely uses 'telltale' (one word, no hyphen) for sail tell-tale. So perhaps earlier name was different - tuft, yarn, ...