Skip to content
This repository has been archived by the owner on Apr 17, 2022. It is now read-only.

upgrade to physfs 2.1 and change location of configuration directory #4693

Closed
wzdev-ci opened this issue Jan 5, 2018 · 39 comments
Closed

upgrade to physfs 2.1 and change location of configuration directory #4693

wzdev-ci opened this issue Jan 5, 2018 · 39 comments

Comments

@wzdev-ci
Copy link
Contributor

wzdev-ci commented Jan 5, 2018

resolution_fixed type_patch (an actual patch, not a request for one) | by Forgon


This patch series ensures compatibility with physfs versions >= 2.1.

Physfs is currently in version 3.0.1 (read the latest release note).

Versions 2.0 and 2.1 replaced several functions (read the deprecated list), 8 of which are part of this game. These are now deprecated and lead to hundreds of compiler warnings:

  • PHYSFS_addToSearchPath
  • PHYSFS_getLastError
  • PHYSFS_getLastModTime
  • PHYSFS_getUserDir
  • PHYSFS_isDirectory
  • PHYSFS_read
  • PHYSFS_removeFromSearchPath
  • PHYSFS_write

The changes are trivial, yet 2 alter game behaviour:

  • function 'PHYSFS_getUserDir()', used to determine a user's home directory, has no equivalent replacement. While easily substituted with 'getenv("HOME")' for Unix, it is no longer available for Windows and Mac OS, where it was a backup to set the location of the configuration directory if the original routine to do so failed. In this case, the configuration directory will now at once be placed in the current directory.

  • mods given in commandline options were accepted if they either existed or were directories. Assuming that mods are placed in archives based on the observation that directories containing mods have no effect, mods are now rejected unless they are regular files, using the 'PHYSFS_Stat' struct which contains metadata about a file or directory. There may be more locations in the source code where 'PHYSFS_exists()' would be better written as 'fileMetaData.filetype == PHYSFS_FILETYPE_REGULAR'.

Many users will have to upgrade physfs and changes to installation scripts will likely be necessary, too.
Nonetheless, I would like to push these overdue changes to the official master branch on Github immediately, unless Per or another developer object.


Issue migrated from trac:4693 at 2022-04-16 12:58:53 -0700

@wzdev-ci
Copy link
Contributor Author

wzdev-ci commented Jan 5, 2018

Forgon uploaded file physfs_add_to_search_path.patch (9.7 KiB)

patch file created with git format-patch (part 1/9)

@wzdev-ci
Copy link
Contributor Author

wzdev-ci commented Jan 5, 2018

Forgon uploaded file physfs_get_last_error.patch (32.3 KiB)

patch file created with git format-patch (part 2/9)

@wzdev-ci
Copy link
Contributor Author

wzdev-ci commented Jan 5, 2018

Forgon uploaded file physfs_get_last_mod_time.patch (1.0 KiB)

patch file created with git format-patch (part 3/9)

@wzdev-ci
Copy link
Contributor Author

wzdev-ci commented Jan 5, 2018

Forgon uploaded file physfs_is_directory.patch (3.5 KiB)

patch file created with git format-patch (part 6/9)

@wzdev-ci
Copy link
Contributor Author

wzdev-ci commented Jan 5, 2018

Forgon uploaded file physfs_read.patch (28.9 KiB)

patch file created with git format-patch (part 7/9)

@wzdev-ci
Copy link
Contributor Author

wzdev-ci commented Jan 5, 2018

Forgon uploaded file physfs_remove_from_search_path.patch (4.7 KiB)

patch file created with git format-patch (part 8/9)

@wzdev-ci
Copy link
Contributor Author

wzdev-ci commented Jan 5, 2018

Forgon uploaded file physfs_write.patch (13.9 KiB)

patch file created with git format-patch (part 9/9)

@wzdev-ci
Copy link
Contributor Author

wzdev-ci commented Jan 5, 2018

Per commented


I think we should release 3.2.4/3.3.0 first. Making such a big platforms-related change just before release seems like a bad idea.

@wzdev-ci
Copy link
Contributor Author

wzdev-ci commented Jan 10, 2018

Forgon commented


Replying to Warzone2100/old-trac-import#4693 (comment:1):

I think we should release 3.2.4/3.3.0 first. Making such a big platforms-related change just before release seems like a bad idea.

The absense of this patch has already caused problems in ArchLinux, which uses physfs-3.* since months.

There, the warzone2100-git package now includes this patch series (replacing a broken patch someone else attempted before), and so will the warzone2100 package with the next release.

@wzdev-ci
Copy link
Contributor Author

wzdev-ci commented Jan 14, 2018

vexed commented


Replying to Warzone2100/old-trac-import#4693 (comment:2):

Replying to Warzone2100/old-trac-import#4693 (comment:1):

I think we should release 3.2.4/3.3.0 first. Making such a big platforms-related change just before release seems like a bad idea.

The absense of this patch has already caused problems in ArchLinux, which uses physfs-3.* since months.

There, the warzone2100-git package now includes this patch series (replacing a broken patch someone else attempted before), and so will the warzone2100 package with the next release.

I agree with Per here, we have been bitten by these type of things before, so it is better to be overly cautious here, since it does change the way we do things now, and a release is sooner rather than later.

If however the release won't be done for months, then it is safe to apply the patch (didn't really look at it yet, but assuming it is correct).

@wzdev-ci
Copy link
Contributor Author

pastdue commented


Are there any major stable distros that are stuck shipping versions of Physfs that are < 2.1?

If so, we can add some code to detect the Physfs version at compile-time and use the existing (old) APIs to maintain compatibility.

@wzdev-ci
Copy link
Contributor Author

wzdev-ci commented Mar 1, 2018

Forgon commented


Replying to Warzone2100/old-trac-import#4693 (comment:4):

Are there any major stable distros that are stuck shipping versions of Physfs that are < 2.1? [...]

Unfortunately, almost all of them.

We could require their users to install a newer version of physfs, but their packages are not allowed to require such a dependency. Some people might never install the game due to such a complication that does not even result in better UX.

Therefore, it is probably best to wait until a reasonable number of distributions have upgraded physfs. Ubuntu 16.0.4.3 LTS will expire in 2021, which is the latest date to consider.

I suggest to investigate which major distributions do not provide packages satisfying the current Qt requirements (please upgrade the Linux Compile Guide if you know them) so that support for systems not meeting that minimum standard could be dropped.

In the meantime, distributions that use a newer version of physfs, like ArchLinux, should use this patch series for packaging.

@wzdev-ci
Copy link
Contributor Author

wzdev-ci commented Mar 1, 2018

pastdue commented


@forgon: Alright, I've gone through and created a modified version of the "safe" subset of your patches here: Warzone2100/warzone2100#143
Patch: https://github.com/Warzone2100/warzone2100/pull/143.patch

Highlighted changes from your patches:

  • Does not include patches that change game behavior (physfs_get_user_dir.patch, mods_regular_files.patch)
  • Maintains compatibility with PhysFS 2.0 (by using new WZ_PhysFS_* wrapper functions when appropriate)

Because of these changes, we should be able to safely apply https://github.com/Warzone2100/warzone2100/pull/143.patch now, resolving all but a single deprecation warning when compiling with PhysFS 2.1+, and maintaining compatibility with PhysFS 2.0.

We can then later investigate the remaining two "behavior changing" patches (physfs_get_user_dir.patch, mods_regular_files.patch).

@wzdev-ci
Copy link
Contributor Author

wzdev-ci commented Mar 7, 2018

Forgon2100 <950973@...> changed status from new to closed

@wzdev-ci
Copy link
Contributor Author

wzdev-ci commented Mar 7, 2018

Forgon2100 <950973@...> changed owner from `` to Forgon2100 <950973@mvrht.com>

@wzdev-ci
Copy link
Contributor Author

wzdev-ci commented Mar 7, 2018

Forgon2100 <950973@...> changed resolution from `` to fixed

@wzdev-ci
Copy link
Contributor Author

wzdev-ci commented Mar 7, 2018

Forgon2100 <950973@...> committed [8]


In Warzone2100/warzone2100@8e97ece:

#CommitTicketReference repository="" revision="8e97eceb5b27bd8bce34f2958ccb635f877b33dc"
fixes #4693: replace deprecated function PHYSFS_addToSearchPath() with function PHYSFS_mount() to avoid compiler warnings with physfs >= 2.0 (patch 1/9)

@wzdev-ci
Copy link
Contributor Author

wzdev-ci commented Mar 7, 2018

past-due <30942300+past-due@...> committed [609]


In Warzone2100/warzone2100@609a9da:

#CommitTicketReference repository="" revision="609a9dabe7f96f73818bd58396d4d8f6fd0110e7"
fixes #4693: replace deprecated function PHYSFS_getLastError()

replace deprecated function PHYSFS_getLastError() with function WZ_PHYSFS_getLastError() to avoid compiler warnings with physfs >= 2.1

@wzdev-ci
Copy link
Contributor Author

wzdev-ci commented Mar 7, 2018

past-due <30942300+past-due@...> committed [862]


In Warzone2100/warzone2100@862bf95:

#CommitTicketReference repository="" revision="862bf9514aa535a5f1791e62a0be244b057b338e"
fixes #4693: replace deprecated function PHYSFS_getLastModTime()

replace deprecated function PHYSFS_getLastModTime() with function WZ_PHYSFS_getLastModTime() to avoid compiler warnings with physfs >= 2.1

Co-Authored-By: Forgon2100 <forgon2100@users.noreply.github.com>

@wzdev-ci
Copy link
Contributor Author

wzdev-ci commented Mar 7, 2018

past-due <30942300+past-due@...> committed [2448]


In Warzone2100/warzone2100@2448e89:

#CommitTicketReference repository="" revision="2448e897797ea97f46430b1f94f2358e5eb965b4"
fixes #4693: replace deprecated function PHYSFS_isDirectory()

replace deprecated function PHYSFS_isDirectory() with function WZ_PHYSFS_isDirectory() to avoid compiler warnings with physfs >= 2.1

Co-Authored-By: Forgon2100 <forgon2100@users.noreply.github.com>

@wzdev-ci
Copy link
Contributor Author

wzdev-ci commented Mar 7, 2018

past-due <30942300+past-due@...> committed [127807027]


In Warzone2100/warzone2100@1278070:

#CommitTicketReference repository="" revision="127807027c8f59d27ac222b990792e94ef346f24"
fixes #4693: replace deprecated function PHYSFS_read()

replace deprecated function PHYSFS_read() with function WZ_PHYSFS_readBytes() to avoid compiler warnings with physfs >= 2.1

Co-Authored-By: Forgon2100 <forgon2100@users.noreply.github.com>

@wzdev-ci
Copy link
Contributor Author

wzdev-ci commented Mar 7, 2018

past-due <30942300+past-due@...> commented


In Warzone2100/warzone2100@ea648d3:

#CommitTicketReference repository="" revision="ea648d31bfab184fc8083adb6837b9de5f78b983"
fixes #4693: replace deprecated function PHYSFS_removeFromSearchPath()

replace deprecated function PHYSFS_removeFromSearchPath() with function WZ_PHYSFS_unmount() to avoid compiler warnings with physfs >= 2.1

Co-Authored-By: Forgon2100 <forgon2100@users.noreply.github.com>

@wzdev-ci
Copy link
Contributor Author

wzdev-ci commented Mar 7, 2018

past-due <30942300+past-due@...> commented


In Warzone2100/warzone2100@ad37dc8:

#CommitTicketReference repository="" revision="ad37dc883d3106ce1c887466a42a0b731e68c588"
fixes #4693: replace deprecated function PHYSFS_write()

replace deprecated function PHYSFS_write() with function WZ_PHYSFS_writeBytes() to avoid compiler warnings with physfs >= 2.1

Co-Authored-By: Forgon2100 <forgon2100@users.noreply.github.com>

@wzdev-ci
Copy link
Contributor Author

wzdev-ci commented Mar 7, 2018

pastdue changed status from closed to reopened

@wzdev-ci
Copy link
Contributor Author

wzdev-ci commented Mar 7, 2018

pastdue changed resolution from fixed to ``

@wzdev-ci
Copy link
Contributor Author

wzdev-ci commented Mar 7, 2018

pastdue commented


I've gone ahead and merged the safe / modified ("no-behavior-change") patches in PR 143, since they do not impact the ability to compile with PhysFS 2.0, but are targeted to prevent a bunch of compilation errors / warnings when attempting to compile with PhysFS 2.1+.

The remaining bits of Forgon's original patch that need to be discussed are physfs_get_user_dir.patch & mods_regular_files.patch, which do contain behavior changes and so have not (yet) been merged.

@wzdev-ci
Copy link
Contributor Author

Forgon commented


The patch [raw-attachment:physfs_mods_regular_files.patch] was updated so that it now works with PhysFS 2.0 (almost) as well as with PhysFS 2.1+.

In addition, it fixes a bug: nonexisting mods were kept in the search path and thus displayed as part of the mod list in the upper left corner of the screen.
To reproduce this problem inside your source code directory execute

mkdir ~/.warzone2100-master/mods/global/nonexistent; ./src/warzone2100 --mod=nonexistent

I also noticed that

  • global mods can be placed directly inside the subdirectory './mods' of the configuration directory, for which I would prefer the subdirectory './mods/global' as their only location
  • campaign mods are broken

@wzdev-ci
Copy link
Contributor Author

Forgon uploaded file physfs_get_user_dir.patch (1.4 KiB)

patch file created with git format-patch (part 9/11)

@wzdev-ci
Copy link
Contributor Author

Forgon uploaded file mods_regular_files.patch (4.2 KiB)

patch file created with git format-patch (part 10/11)

@wzdev-ci
Copy link
Contributor Author

Forgon uploaded file physfs_changelog.patch (2.2 KiB)

patch file created with git format-patch (part 11/11)

@wzdev-ci
Copy link
Contributor Author

Forgon2100 <forgon2100@...> changed status from reopened to closed

@wzdev-ci
Copy link
Contributor Author

Forgon2100 <forgon2100@...> changed resolution from `` to fixed

@wzdev-ci
Copy link
Contributor Author

Forgon2100 <forgon2100@...> committed [827]


In Warzone2100/warzone2100@827cd59:

#CommitTicketReference repository="" revision="827cd597385aa34f77c3ab45f7249b68799114b9"
fixes #4693: replace deprecated function PHYSFS_getUserDir() with getenv("HOME"), for Unix only, to avoid compiler warnings with physfs >= 2.1 (patch 9/11)

@wzdev-ci
Copy link
Contributor Author

Forgon2100 <forgon2100@...> committed [0]


In Warzone2100/warzone2100@0d33e57:

#CommitTicketReference repository="" revision="0d33e57b495e7f6b6fb4e793db098b1c02198893"
fixes #4693: mods given in commandline options are no longer accepted if either existing or directories, but rejected unless they are regular files (patch 10/11)

@wzdev-ci
Copy link
Contributor Author

Forgon2100 <forgon2100@...> committed [8]


In Warzone2100/warzone2100@8b82c5c:

#CommitTicketReference repository="" revision="8b82c5c8d348137bb9d880a07d7e338183e6fdd8"
fixes #4693: updated ChangeLog (patch 11/11)

@wzdev-ci
Copy link
Contributor Author

Forgon commented


Pastdue amended this patch series with a further commit, which changed the location of the home directory:

`
commit 54901bcd1672c9e22db35c66741ec2eda123b491
Author: past-due 30942300+past-due@users.noreply.github.com
Date: Wed Mar 21 15:57:52 2018 -0400

More robust handling of getting the app user-data (prefs) dir

- Fixed: On Windows, applications should never write application-data files to My Documents (`CSIDL_PERSONAL`) - instead, use `%APPDATA%\org\appname`.
- Fixed: By using `PHYSFS_getPrefDir` when available (PhysFS >= 2.1), we include proper support for numerous other operating systems / configurations.
- Fixed: Fallback when `PHYSFS_getPrefDir` is not available (PhysFS < 2.1) that matches its behavior as closely as possible.
- Fixed: Crash introduced by commit: 827cd597385aa34f77c3ab45f7249b68799114b9 when `getenv("HOME")` returns NULL.

Changes of note:
- On Windows the application-data path has changed to the appropriate location under `%APPDATA%\Warzone 2100 Project\Warzone 2100 <version>`
- On Linux / Unix, the warzone application-data path follows the [XDG base directory spec](https://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html) the same way that `PHYSFS_getPrefDir` does.
See the updated `README.md` for details.

'

It introduced the following compiler warning for physfs 2.1+ users:

main.cpp:189:20: error: ‘std::__cxx11::string getPlatformPrefDir_Fallback(const char*, const char*)’ defined but not used [-Werror=unused-function]
 static std::string getPlatformPrefDir_Fallback(const char *org, const char *app)
                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1plus: all warnings being treated as errors

The problem is now fixed.

@wzdev-ci
Copy link
Contributor Author

Forgon changed title from upgrade to physfs 2.1 to upgrade to physfs 2.1 and change location of configuration directory

@wzdev-ci
Copy link
Contributor Author

Forgon uploaded file physfs_compile_warning.patch (3.3 KiB)

patch file created with git format-patch (part 13/13)

@wzdev-ci
Copy link
Contributor Author

Forgon2100 <forgon2100@...> committed [0675]


In Warzone2100/warzone2100@0675a20:

#CommitTicketReference repository="" revision="0675a20be7b861737573f1c6cd8958ba1ee087c6"
fixes #4693: fix compiler warning for physfs >= 2.1 and updated ChangeLog (patch 13/13)

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

1 participant