Build Zeos DCUs from the command line?

[aehimself]

Hello,

I am using Zeos 7.3 by DCUs. As the source of 7.3 is changing quickly (Thanks to the devs!!!) It's getting a bit slow to manually build each package for Win32 Debug, Win32 Release, Win64 Debug, Win64 Release so I decided to write a script for it.
I can compile / build the first package (ZCore.dpk) for all 4 targets, .DCUs are going to the right places. But when attempting to build the second package (ZPlain.dpk) it fails with the error:

Embarcadero Delphi for Win32 compiler version 33.0
Copyright (c) 1983,2018 Embarcadero Technologies, Inc.
C:\LocalWork\_DelphiComponents\Zeos\src\plain\ZOleDB.pas(71) Fatal: E2213 Bad packaged unit format: ZCore.dcp.ZCompatibility - Expected version: 33.0, Windows Unicode(x86) Found version: 33.0, Windows Unicode(x64)

This tells me that the previous build (ZCore Win64 Release) is putting something to the wrong place OOOOOR the Win32 build is not looking for it's files at the right places.

The command lines I'm using:
%DCC32% %DEBUGPARAM% %COMMONPARAM% -R%ZEOSPATH%\Win32\Debug -U"%DELPHIPATH%\lib\Win32\debug" -I%ZEOSPATH%\Win32\Debug -E.\Win32\Debug -NU%ZEOSPATH%\Win32\Debug --description:%1 -NO%ZEOSPATH%\Win32\Debug %2 > %TMPFILE% 2>&1
%DCC32% %RELEASEPARAM% %COMMONPARAM% -R%ZEOSPATH%\Win32\Release -U"%DELPHIPATH%\lib\Win32\release" -I%ZEOSPATH%\Win32\Release -E.\Win32\Release -NU%ZEOSPATH%\Win32\Release --description:%1 -NO%ZEOSPATH%\Win32\Release %2 > %TMPFILE% 2>&1
%DCC64% %DEBUGPARAM% %COMMONPARAM% -R%ZEOSPATH%\Win64\Debug -U"%DELPHIPATH%\lib\Win64\debug" -I%ZEOSPATH%\Win64\Debug -E.\Win64\Debug -NU%ZEOSPATH%\Win64\Debug --description:%1 -NO%ZEOSPATH%\Win64\Debug %2 > %TMPFILE% 2>&1
%DCC64% %RELEASEPARAM% %COMMONPARAM% -R%ZEOSPATH%\Win64\Release -U"%DELPHIPATH%\lib\Win64\Release" -I%ZEOSPATH%\Win64\Release -E.\Win32\Release -NU%ZEOSPATH%\Win64\Release --description:%1 -NO%ZEOSPATH%\Win64\Release %2 > %TMPFILE% 2>&1

Where variables are:
SET DELPHIPATH=C:\Program Files (x86)\Embarcadero\Studio\20.0
SET DELPHIUSERPATH=C:\Users\Public\Documents\Embarcadero\Studio\20.0
SET DCC32="%DELPHIPATH%\Bin\dcc32.exe"
SET DCC64="%DELPHIPATH%\Bin\dcc64.exe"
SET ZEOSPATH=C:\LocalWork\_DelphiComponents\Zeos\DCU
SET DEBUGPARAM=-$O- -$W+ -$R+ -Q+ -DDEBUG -V -VN -VR
SET RELEASEPARAM=-$D0 -$L- -$Y- -DRELEASE
SET WIN32PARAM=-LE%DELPHIUSERPATH%\Bpl -LN%DELPHIUSERPATH%\Dcp -NH%DELPHIUSERPATH%\hpp\Win32 -NB%DELPHIUSERPATH%\Dcp -O%DELPHIUSERPATH%\Dcp
SET WIN64PARAM=-LE%DELPHIUSERPATH%\Bpl\Win64 -LN%DELPHIUSERPATH%\Dcp\Win64 -NH%DELPHIUSERPATH%\hpp\Win64 -NB%DELPHIUSERPATH%\Dcp\Win64 -O%DELPHIUSERPATH%\Dcp\Win64
SET COMMONPARAM=--no-config --legacy-ifend -B -Q -TX.bpl -NSWinapi;System;Data

What am I missing? I already made sure no rogue versions of the .DCUs are located on the drive anywhere. Also worth to note that I don't have much experience with command line building, so please be gentle

[marsupilami]

Sorry - I didn't look at your build command (yet). It seems a lot to understand

Maybe this helps anyway. Delphi supports building projects using msbuild since Delphi 2009. The following is what I call for building the ztestall project on Jenkins:

Code: Select all

call "C:\Program Files (x86)\Embarcadero\RAD Studio\9.0\bin\rsvars.bat"
C:\Windows\Microsoft.NET\Framework\v4.0.30319\msbuild.exe /t:build /p:platform=Win64 /p:config=Release /p:UnitSearchPath="%BDS%\lib\win64\release;..\..\src\core;..\..\src\parsesql;..\..\src\plain;..\..\src\dbc;..\..\src\component;..\..\test\bugreport;..\..\test\component;..\..\test\config;;..\..\test\core;..\..\test\dbc;..\..\test\external;..\..\test\framework;..\..\test\parsesql;..\..\test\plain;..\..\test\performance" zeos\packages\DelphiXE2\ZTestAll.dproj

• rsvars is a batch file from Embarcadero that sets the path and all other environment variables. It seems to exist like forever.
• /t is the target - what you want to do. In this case I want to build the project.
• /p specifies additional parameters like the platform (Win32, Win64, ...), configuration is the project configuration to use (Debug, Release, ...), UnitSearchPath overrides the unit search path. If you specify that one, note that it will not extend the unit search path but replace it. Maybe you don't want to do that and everything will be simple for you

Unfortunately there is no extensive documentation on this. Maybe this is a starting point: "MSBUILD" in the delphi online docs. I needed quite some googlefoo and time for testing to find this. You could combine this with Jenkins to get your own automated Zeos builds running

[aehimself]

I am almost certain I'm missing a path declaration somewhere. When I start my script and it fails, I can not even build the package from the Delphi IDE anymore; I had to remove all z*.dcu files from Users\Public\Documents\...
Or I left one path declaration in when building 64 bit versions.

[marsupilami]

I had to remove all z*.dcu files from Users\Public\Documents\...

Erm - why do dcu files go to Users\Public\Documents? Zeos packages usually are preconfigured to put dcu files into ".\$(Platform)\$(Config)" - which should create a subdirectory in the Zeos package directory for each architecture? This way dcu files for different architectures get separated. I am not sure, how Delphi separates bpl files and dcp files though.

Note: I usually don't bother with the dcu files. I configure an dcu output path for my application and include the source paths for Zeos. This way Zeos gets rebuilt by every application as soon as I rebuild that application. The tradeoff is that this requires a bit more time during a rebuild / recompile.

[aehimself]

Typo... dcp

C:\Users\Public\Documents\Embarcadero\Studio\20.0\DCP

[Fr0sT]

Building projects for Delphis > 7 really should be done via MSBuild. I made some build scripts here https://github.com/Fr0sT-Brutal/Delphi_BuildScripts that I use from time to time.

[aehimself]

Holy f...

So my batch is:

Code: Select all

@ECHO OFF
CALL "C:\Program Files (x86)\Embarcadero\Studio\20.0\bin\rsvars.bat"
CALL :BUILD ZCore
CALL :BUILD ZPlain
CALL :BUILD ZParseSQL
CALL :BUILD ZDBC
CALL :BUILD ZComponent
CALL :BUILD ZComponentDesign
GOTO :END

:BUILD
CALL :MSBUILD %~1.dproj Win32 Debug
CALL :MSBUILD %~1.dproj Win32 Release
CALL :MSBUILD %~1.dproj Win64 Debug
CALL :MSBUILD %~1.dproj Win64 Release
GOTO :eof

:MSBUILD
ECHO Building %1 as platform %2, configuration %3...
C:\Windows\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe /t:build /nologo /verbosity:q /property:Platform=%2;Config=%3;DCC_DCUOutput=C:\LocalWork\_DelphiComponents\Zeos\DCU\%2\%3 "%~1"
IF ERRORLEVEL 1 PAUSE
GOTO :eof

:END

And it works... Idk why the built-in DCC32-64 was a mess, with MSBuild it's clean and nice... Emba did it again, I suppose. I'll add some proper error handling and not to attempt to build the design package in 64 bit.

Thanks

[aehimself]

So not everything is pure joy, unfortunately I just recompiled all DCUs with the script above; but when I attempt to run a program using these, I am greeted with the error:

[dcc32 Fatal Error] uMySQLTable.pas(5): F2051 Unit ZDbcPostgreSqlUtils was compiled with a different version of ZPlainPostgreSqlDriver.TZPostgreSQLFieldCode

Seems I'll have to stick to the manual mode

[Fr0sT]

Maybe you have setting in program project pointing to some sources? Try renaming source folder.

[aehimself]

@Frost,

I just doublechecked, I am not referring to anything specific in the project options for the application; only the general Delphi settings are changed as following:
- Debugger \ Embarcadero debuggers \ Debug Source Path: [...]$(DCOMP)\Zeos\src\component;$(DCOMP)\Zeos\src\core;$(DCOMP)\Zeos\src\dbc;$(DCOMP)\Zeos\src\parsesql;$(DCOMP)\Zeos\src\plain[...]
- Language \ Delphi \ Library \ Library path (changing the folder for Win32 and Win64 of course):
[...]$(DCOMP)\Zeos\DCU\Win64\release[...]
- Language \ Delphi \ Library \ Browsing path:
[...]$(DCOMP)\Zeos\src\Component;$(DCOMP)\Zeos\src\Core;$(DCOMP)\Zeos\src\Dbc;$(DCOMP)\Zeos\src\ParseSQL;$(DCOMP)\Zeos\src\plain[...]
- Language \ Delphi \ Library \ Debug DCU path (changing the folder for Win32 and Win64 of course):
[...]$(DCOMP)\Zeos\DCU\Win64\debug[...]

In the Zeos project, I changed only the following in Project Options for each package:
- Building \ Delphi Compiler \ Compiling \ Unit output directory:
$(DCOMP)\Zeos\DCU\$(Platform)\$(Config)

When I was building the .DCUs wit msbuild, I paused the execution after every platform and config so I could ensure that the .DCUs indeed are going to the right folder. I also confirmed that the sizes of debug .DCUs are larger than release ones, also Win64 ones are larger than Win32 ones. So I indeed believe so that .DCUs are in the right place.

If I could solve this, I could automate refreshing the GIT repository and rebuilding the .DCUs if needed. And I would love that!

[aehimself]

So, interesting thing. Release configuration of applications using the .DCUs built with MSBuild do work, only Debug mode does not.
If I change the order, building Release .DCUs first, the debug configuration of applications will compile, Release will fail.

[aehimself]

HOLY F*CK IT WORKS!!!! Turns out I only declared DCC_DCUOutput, where a couple more was needed. Just change the variables to your needs, and feel free to use:

Code: Select all

@ECHO OFF
SET BDSVERSION=21.0
SET DCOMMON=C:\Users\Public\Documents\Embarcadero\Studio\%BDSVERSION%
SET ZEOSDCU=C:\LocalWork\_DelphiComponents\Zeos\DCU
SET ZEOSPACKAGE=C:\LocalWork\_DelphiComponents\Zeos\packages\DelphiXE10.4

CD /D %ZEOSPACKAGE%
RMDIR /S /Q %ZEOSDCU%
CALL "C:\Program Files (x86)\Embarcadero\Studio\%BDSVERSION%\bin\rsvars.bat"
CALL :BUILDALL ZCore
CALL :BUILDALL ZPlain
CALL :BUILDALL ZParseSQL
CALL :BUILDALL ZDBC
CALL :BUILDALL ZComponent
CALL :BUILD32 ZComponentDesign
GOTO :END

:BUILDALL
CALL :BUILD32 %~1
CALL :BUILD64 %~1
CALL :BUILDLINUX %~1
GOTO :eof

:BUILD32
CALL :MSBUILD %~1.dproj Win32 Debug
CALL :MSBUILD %~1.dproj Win32 Release
GOTO :eof

:BUILD64
CALL :MSBUILD %~1.dproj Win64 Debug
CALL :MSBUILD %~1.dproj Win64 Release
GOTO :eof

:BUILDLINUX
CALL :MSBUILD %~1.dproj Linux64 Debug
CALL :MSBUILD %~1.dproj Linux64 Release
GOTO :eof

:MSBUILD
ECHO Building %1 as platform %2, configuration %3...
SET NOWARN=/consoleloggerparameters:ErrorsOnly
SET SUFFIX=\%2
IF %2==Win32 (
 SET SUFFIX=
 IF %3==Debug SET NOWARN=
)
%FRAMEWORKDIR%\MSBuild.exe /t:build /nologo /verbosity:q %NOWARN% /property:Platform=%2;Config=%3;DCC_DCUOutput=%ZEOSDCU%\%2\%3;DCC_BPLOutput=%DCOMMON%\Bpl%SUFFIX%;DCC_DCPOutput=%DCOMMON%\Dcp%SUFFIX%;DCC_HPPOutput=%DCOMMON%\Hpp\%2 "%~1"
GOTO :eof

:END
PAUSE

P.s.: Zeos currently does not compile on Linux ;)

ZPlainLoader.pas(197): error E2003: Undeclared identifier: 'dlopen' [ZPlain.dproj]
ZPlainLoader.pas(197): error E2003: Undeclared identifier: 'RTLD_GLOBAL' [ZPlain.dproj]
ZPlainDriver.pas(705): error F2063: Could not compile used unit 'ZPlainLoader.pas' [ZPlain.dproj]

[marsupilami]

Unfortunately I don't have a Delphi Edition that has the Linux compiler. Which Error do you get?

[aehimself]

ZPlainLoader.pas(197): error E2003: Undeclared identifier: 'dlopen' [ZPlain.dproj]
ZPlainLoader.pas(197): error E2003: Undeclared identifier: 'RTLD_GLOBAL' [ZPlain.dproj]
ZPlainDriver.pas(705): error F2063: Could not compile used unit 'ZPlainLoader.pas' [ZPlain.dproj]

At least this is what MSBuild says. I did not have plans to use Zeos on Linux yet, but since .DCU building finally seems to work I decided why not...
Once I get home (abroad for a couple of days atm) I'll check if the IDE gives the same results. It's possible that I am missing some paths only - Delphi did not even add it's own Linux64 folder to the Library path.

[marsupilami]

Hmmm - to me it looks like our include file isn't prepared to work on Delphi for Linux.

ZPlainLoader.pas(197): error E2003: Undeclared identifier: 'dlopen' [ZPlain.dproj]

This line basically says that it tries to use dlopen which is the Linux version of LoadLibrary but doesn't find it. So the question would be which function gets used on Delphi for Linux to load an external library and then in which unit it is declared. Mabe the way this gets handeled on OS X can be a hint because they are a unixoid system too.