Issues related to using FormatSettings.LongDateFormat for Timestamps in ZDatasetParams

[MJFShark]

Zeoslib seems to use FormatSettings.LongDateFormat in the ZDatasetparam unit to convert timestamps. This appears to be incorrect (or at least the default LongDateFormat format string is currently actually a very verbose date only format, perhaps that changed at some point?) On my system it defaults to 'dddd, mmmm d, yyyy' and includes no time component. This can be worked around by doing something like:

FormatSettings.LongDateFormat := FormatSettings.ShortDateFormat + ' ' + FormatSettings.LongTimeFormat;

But it seems it shouldn't be necessary. This comes into play if you set a timestamp param to a string in order to convert it.

ZParam := TZParam.Create(nil);
ZParam.Datatype := ftdatetime;
ZParam.Value := '1903. 03. 02.';

On my system which has a longdateformat as shown above, this goes into an infinite loop. Obviously this is a bad value (It's from a recent csv file used for debugging purposes) but it would be best if it just failed instead of looping.

The eventual call that fails is:

Test := TryUniToTimeStamp('1903. 03. 02.', 13, 'dddd, mmmm d, yyyy', TS);
(I just mention this as it's the smallest code to show the issue.)

I can enter this as a bug in the tracker, but I wanted others to check to see if maybe I'm just misunderstanding something. Thanks as always!

-Mark

[aehimself]

It's important here to mention the IDE used. Delphi 11 changed how date parsing is done and it caused our software at work to start failing at almost any point.

Edit: 2 things.

1, Infinite loop? If I'm trying to feed it with a value surely different from my LongDateFormat, it simply crashes with a range check error.
2, You are right with .LongDateFormat. Even Delphi RTL is using ShortDateFormat for string to date(time) transformation:

Code: Select all

function ScanDate(const S: string; var Pos: Integer; var Date: TDateTime;
  const AFormatSettings: TFormatSettings): Boolean; overload;
[...]
begin
  DateSeq := ParseDateTimeFormat(AFormatSettings.ShortDateFormat, False);
[...]

I vote on chaning this to .ShortDayFormat at least, however it won't solve your issue just yet.

Edit-edit: it'll clearly be an issue with Zeos. Check InternalSetAsUnicodeString:

stDate: if (FDataSet = nil) or not TZAbstractRODataset(FDataSet).FormatSettings.EditDateFormatSettings.TryStrToDate(PZDate(DataAddr)^, Value) then
if not TryUniToDate(P, L, {$IFDEF WITH_FORMATSETTINGS}FormatSettings{$ELSE}SysUtils{$ENDIF}.ShortDateFormat, PZDate(DataAddr)^) then goto jmpErr;
stTime: if (FDataSet = nil) or not TZAbstractRODataset(FDataSet).FormatSettings.EditTimeFormatSettings.TryStringToTime(PZTime(DataAddr)^, Value, 9) then
if not TryUniToTime(P, L, {$IFDEF WITH_FORMATSETTINGS}FormatSettings{$ELSE}SysUtils{$ENDIF}.LongTimeFormat, PZTime(DataAddr)^) then goto jmpErr;
stTimestamp: if (FDataSet = nil) or not TZAbstractRODataset(FDataSet).FormatSettings.EditTimestampFormatSettings.TryStrToTimestamp(PZTimestamp(DataAddr)^, Value, 9) then
if not TryUniToTimeStamp(P, L, {$IFDEF WITH_FORMATSETTINGS}FormatSettings{$ELSE}SysUtils{$ENDIF}.LongDateFormat, PZTimeStamp(DataAddr)^) then goto jmpErr;

If no dataset is assigned, then it uses .ShortDateFormat for date only parsing and .LongDateFormat for time and date/time parsing. This is clearly wrong.
I'll check what it does if we have a dataset assigned.

Edit-edit-edit:
Same thing with a dataset. However an other suspicious line in TryUniToDate:

Code: Select all

            else if (F in [Byte('-'),Byte('/'),Byte('\'),Byte(' ')]) then begin

Shouldn't we be using FormatSettings.DateSeparator instead of a constant array...?

[MJFShark]

That's interesting. For me, this call goes into an infinite loop in both 32bit and 64bit debug mode. Delphi 11.3 patch 1.

Test := TryUniToTimeStamp('1903. 03. 02.', 13, 'dddd, mmmm d, yyyy', TS);

That's odd that it causes a range error for you instead, but I haven't spent a lot of time debugging it. On a side note, I'm not sure I've ever seen a date formatted that way before. Is that a regional format in some country?

-Mark

[MJFShark]

Update! I've also now seen it give range check/pointer errors when trying to convert that format. It looks like (and confirmed from the thread you linked to) that those library functions just can't handle multi-char date delimiters. I'm not sure how important that is, since that's a format I've never seen "in the wild." The general issue about using LongDateFormat for timestamps still should be looked at.

-Mark

[aehimself]

Yes, this is the standard date format in Hungary (where I live).

I don't know which library you mean, because built-in RTL functions do handle "multi-char" delimiters (up to 2, that is) fine. The only issue on D11+ was that it's not accepting "2023. 06. 17" because it requires the trailing delimiter as well as short date format is "yyyy/mm/dd/".

[Fr0sT]

However an other suspicious line in TryUniToDate:

Code: Select all

            else if (F in [Byte('-'),Byte('/'),Byte('\'),Byte(' ')]) then begin

Shouldn't we be using FormatSettings.DateSeparator instead of a constant array...?

I guess TryUniToTimeStamp is meant to be a kind of universal converter regardless of some regional settings to allow more broad set of formats on input. Thus these separators. Multichar separators were likely not foreseen. As you can see, this function does custom parsing with whole lot of black magic to boost performance so has nothing to do with current locale or RTL.
Anyway I consider the practice of assigning a string value to a timestamp field toxic and only allowable for simple apps by beginners. Serious apps should convert dates themselves according to locale settings and assign TDate(Time) values.