From df5218b4c30b2fa1ba0d1ada4166ad85bc83be26 Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Wed, 13 Jan 2016 14:30:53 +0100 Subject: config.mak.uname: support MSys2 For a long time, Git for Windows lagged behind Git's 2.x releases because the Git for Windows developers wanted to let that big jump coincide with a well-needed jump away from MSys to MSys2. To understand why this is such a big issue, it needs to be noted that many parts of Git are not written in portable C, but instead Git relies on a POSIX shell and Perl to be available. To support the scripts, Git for Windows has to ship a minimal POSIX emulation layer with Bash and Perl thrown in, and when the Git for Windows effort started in August 2007, this developer settled on using MSys, a stripped down version of Cygwin. Consequently, the original name of the project was "msysGit" (which, sadly, caused a *lot* of confusion because few Windows users know about MSys, and even less care). To compile the C code of Git for Windows, MSys was used, too: it sports two versions of the GNU C Compiler: one that links implicitly to the POSIX emulation layer, and another one that targets the plain Win32 API (with a few convenience functions thrown in). Git for Windows' executables are built using the latter, and therefore they are really just Win32 programs. To discern executables requiring the POSIX emulation layer from the ones that do not, the latter are called MinGW (Minimal GNU for Windows) when the former are called MSys executables. This reliance on MSys incurred challenges, too, though: some of our changes to the MSys runtime -- necessary to support Git for Windows better -- were not accepted upstream, so we had to maintain our own fork. Also, the MSys runtime was not developed further to support e.g. UTF-8 or 64-bit, and apart from lacking a package management system until much later (when mingw-get was introduced), many packages provided by the MSys/MinGW project lag behind the respective source code versions, in particular Bash and OpenSSL. For a while, the Git for Windows project tried to remedy the situation by trying to build newer versions of those packages, but the situation quickly became untenable, especially with problems like the Heartbleed bug requiring swift action that has nothing to do with developing Git for Windows further. Happily, in the meantime the MSys2 project (https://msys2.github.io/) emerged, and was chosen to be the base of the Git for Windows 2.x. Just like MSys, MSys2 is a stripped down version of Cygwin, but it is actively kept up-to-date with Cygwin's source code. Thereby, it already supports Unicode internally, and it also offers the 64-bit support that we yearned for since the beginning of the Git for Windows project. MSys2 also ported the Pacman package management system from Arch Linux and uses it heavily. This brings the same convenience to which Linux users are used to from `yum` or `apt-get`, and to which MacOSX users are used to from Homebrew or MacPorts, or BSD users from the Ports system, to MSys2: a simple `pacman -Syu` will update all installed packages to the newest versions currently available. MSys2 is also *very* active, typically providing package updates multiple times per week. It still required a two-month effort to bring everything to a state where Git's test suite passes, many more months until the first official Git for Windows 2.x was released, and a couple of patches still await their submission to the respective upstream projects. Yet without MSys2, the modernization of Git for Windows would simply not have happened. This commit lays the ground work to supporting MSys2-based Git builds. Assisted-by: Waldek Maleska Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano diff --git a/config.mak.uname b/config.mak.uname index f34dcaa..b0592c1 100644 --- a/config.mak.uname +++ b/config.mak.uname @@ -518,7 +518,7 @@ ifneq (,$(findstring MINGW,$(uname_S))) NO_INET_NTOP = YesPlease NO_POSIX_GOODIES = UnfortunatelyYes DEFAULT_HELP_FORMAT = html - COMPAT_CFLAGS += -D__USE_MINGW_ACCESS -D_USE_32BIT_TIME_T -DNOGDI -Icompat -Icompat/win32 + COMPAT_CFLAGS += -D_USE_32BIT_TIME_T -DNOGDI -Icompat -Icompat/win32 COMPAT_CFLAGS += -DSTRIP_EXTENSION=\".exe\" COMPAT_OBJS += compat/mingw.o compat/winansi.o \ compat/win32/pthread.o compat/win32/syslog.o \ @@ -541,8 +541,25 @@ ifneq (,$(wildcard ../THIS_IS_MSYSGIT)) INTERNAL_QSORT = YesPlease HAVE_LIBCHARSET_H = YesPlease NO_GETTEXT = YesPlease + COMPAT_CLFAGS += -D__USE_MINGW_ACCESS else - NO_CURL = YesPlease + ifeq ($(shell expr "$(uname_R)" : '2\.'),2) + # MSys2 + CC = gcc + prefix = /mingw32 + COMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 + INSTALL = /bin/install + NO_R_TO_GCC_LINKER = YesPlease + INTERNAL_QSORT = YesPlease + HAVE_LIBCHARSET_H = YesPlease + NO_GETTEXT = YesPlease + USE_LIBPCRE= YesPlease + NO_CURL = + USE_NED_ALLOCATOR = YesPlease + else + COMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO + NO_CURL = YesPlease + endif endif endif ifeq ($(uname_S),QNX) -- cgit v0.10.2-6-g49f6 From 7b40ae86a3f51080ed3550b005fc974024b208fe Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Wed, 13 Jan 2016 14:31:01 +0100 Subject: config.mak.uname: supporting 64-bit MSys2 This just makes things compile, the test suite needs extra tender loving care in addition to this change. We will address these issues in later commits. While at it, also allow building MSys2 Git (i.e. a Git that uses MSys2's POSIX emulation layer). Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano diff --git a/config.mak.uname b/config.mak.uname index b0592c1..4b2e1b8 100644 --- a/config.mak.uname +++ b/config.mak.uname @@ -518,13 +518,12 @@ ifneq (,$(findstring MINGW,$(uname_S))) NO_INET_NTOP = YesPlease NO_POSIX_GOODIES = UnfortunatelyYes DEFAULT_HELP_FORMAT = html - COMPAT_CFLAGS += -D_USE_32BIT_TIME_T -DNOGDI -Icompat -Icompat/win32 + COMPAT_CFLAGS += -DNOGDI -Icompat -Icompat/win32 COMPAT_CFLAGS += -DSTRIP_EXTENSION=\".exe\" COMPAT_OBJS += compat/mingw.o compat/winansi.o \ compat/win32/pthread.o compat/win32/syslog.o \ compat/win32/dirent.o BASIC_CFLAGS += -DPROTECT_NTFS_DEFAULT=1 - BASIC_LDFLAGS += -Wl,--large-address-aware EXTLIBS += -lws2_32 GITLIBS += git.res PTHREAD_LIBS = @@ -545,8 +544,17 @@ ifneq (,$(wildcard ../THIS_IS_MSYSGIT)) else ifeq ($(shell expr "$(uname_R)" : '2\.'),2) # MSys2 + prefix = /usr/ + ifeq (MINGW32,$(MSYSTEM)) + prefix = /mingw32 + endif + ifeq (MINGW64,$(MSYSTEM)) + prefix = /mingw64 + else + COMPAT_CFLAGS += -D_USE_32BIT_TIME_T + BASIC_LDFLAGS += -Wl,--large-address-aware + endif CC = gcc - prefix = /mingw32 COMPAT_CFLAGS += -D__USE_MINGW_ANSI_STDIO=0 INSTALL = /bin/install NO_R_TO_GCC_LINKER = YesPlease -- cgit v0.10.2-6-g49f6 From 9e2af084d4326ea2b1192074fb1d1ccefe0016a3 Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Thu, 14 Jan 2016 17:51:54 +0100 Subject: nedmalloc: allow compiling with MSys2's compiler With MSys2's GCC, `ReadWriteBarrier` is already defined, and FORCEINLINE unfortunately gets defined incorrectly. Let's work around both problems, using the MSys2-specific __MINGW64_VERSION_MAJOR constant to guard the FORCEINLINE definition so as not to affect other platforms. Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano diff --git a/compat/nedmalloc/malloc.c.h b/compat/nedmalloc/malloc.c.h index f216a2a..bf70037 100644 --- a/compat/nedmalloc/malloc.c.h +++ b/compat/nedmalloc/malloc.c.h @@ -720,6 +720,9 @@ struct mallinfo { inlining are defined as macros, so these aren't used for them. */ +#ifdef __MINGW64_VERSION_MAJOR +#undef FORCEINLINE +#endif #ifndef FORCEINLINE #if defined(__GNUC__) #define FORCEINLINE __inline __attribute__ ((always_inline)) @@ -1382,6 +1385,7 @@ LONG __cdecl _InterlockedExchange(LONG volatile *Target, LONG Value); /*** Atomic operations ***/ #if (__GNUC__ * 10000 + __GNUC_MINOR__ * 100 + __GNUC_PATCHLEVEL__) > 40100 + #undef _ReadWriteBarrier #define _ReadWriteBarrier() __sync_synchronize() #else static __inline__ __attribute__((always_inline)) long __sync_lock_test_and_set(volatile long * const Target, const long Value) -- cgit v0.10.2-6-g49f6 From 3ecd153a3bdbefb25a4bf451d7b91cf8f1ece2a9 Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Thu, 14 Jan 2016 17:51:59 +0100 Subject: compat/mingw: support MSys2-based MinGW build The excellent MSys2 project brings a substantially updated MinGW environment including newer GCC versions and new headers. To support compiling Git, let's special-case the new MinGW (tell-tale: the _MINGW64_VERSION_MAJOR constant is defined). Note: this commit only addresses compile failures, not compile warnings (that task is left for a future patch). Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano diff --git a/compat/mingw.h b/compat/mingw.h index 57ca477..10e2e4c 100644 --- a/compat/mingw.h +++ b/compat/mingw.h @@ -1,14 +1,26 @@ +#ifdef __MINGW64_VERSION_MAJOR +#include +#include +typedef _sigset_t sigset_t; +#endif #include #include +/* MinGW-w64 reports to have flockfile, but it does not actually have it. */ +#ifdef __MINGW64_VERSION_MAJOR +#undef _POSIX_THREAD_SAFE_FUNCTIONS +#endif + /* * things that are not available in header files */ -typedef int pid_t; typedef int uid_t; typedef int socklen_t; +#ifndef __MINGW64_VERSION_MAJOR +typedef int pid_t; #define hstrerror strerror +#endif #define S_IFLNK 0120000 /* Symbolic link */ #define S_ISLNK(x) (((x) & S_IFMT) == S_IFLNK) @@ -100,8 +112,10 @@ static inline int symlink(const char *oldpath, const char *newpath) { errno = ENOSYS; return -1; } static inline int fchmod(int fildes, mode_t mode) { errno = ENOSYS; return -1; } +#ifndef __MINGW64_VERSION_MAJOR static inline pid_t fork(void) { errno = ENOSYS; return -1; } +#endif static inline unsigned int alarm(unsigned int seconds) { return 0; } static inline int fsync(int fd) @@ -176,8 +190,10 @@ int pipe(int filedes[2]); unsigned int sleep (unsigned int seconds); int mkstemp(char *template); int gettimeofday(struct timeval *tv, void *tz); +#ifndef __MINGW64_VERSION_MAJOR struct tm *gmtime_r(const time_t *timep, struct tm *result); struct tm *localtime_r(const time_t *timep, struct tm *result); +#endif int getpagesize(void); /* defined in MinGW's libgcc.a */ struct passwd *getpwuid(uid_t uid); int setitimer(int type, struct itimerval *in, struct itimerval *out); @@ -301,8 +317,10 @@ static inline int getrlimit(int resource, struct rlimit *rlp) /* * Use mingw specific stat()/lstat()/fstat() implementations on Windows. */ +#ifndef __MINGW64_VERSION_MAJOR #define off_t off64_t #define lseek _lseeki64 +#endif /* use struct stat with 64 bit st_size */ #ifdef stat @@ -375,8 +393,12 @@ static inline char *mingw_find_last_dir_sep(const char *path) int mingw_offset_1st_component(const char *path); #define offset_1st_component mingw_offset_1st_component #define PATH_SEP ';' +#ifndef __MINGW64_VERSION_MAJOR #define PRIuMAX "I64u" #define PRId64 "I64d" +#else +#include +#endif void mingw_open_html(const char *path); #define open_html mingw_open_html -- cgit v0.10.2-6-g49f6 From 466931d9e10017aca5548f23e3efbcb3063d3c34 Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Thu, 14 Jan 2016 17:52:02 +0100 Subject: compat/winansi: support compiling with MSys2 MSys2 already defines the _CONSOLE_FONT_INFOEX structure. Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano diff --git a/compat/winansi.c b/compat/winansi.c index ceff55b..3aa5535 100644 --- a/compat/winansi.c +++ b/compat/winansi.c @@ -23,6 +23,7 @@ static HANDLE hthread, hread, hwrite; static HANDLE hconsole1, hconsole2; #ifdef __MINGW32__ +#if !defined(__MINGW64_VERSION_MAJOR) || __MINGW64_VERSION_MAJOR < 5 typedef struct _CONSOLE_FONT_INFOEX { ULONG cbSize; DWORD nFont; @@ -32,6 +33,7 @@ typedef struct _CONSOLE_FONT_INFOEX { WCHAR FaceName[LF_FACESIZE]; } CONSOLE_FONT_INFOEX, *PCONSOLE_FONT_INFOEX; #endif +#endif typedef BOOL (WINAPI *PGETCURRENTCONSOLEFONTEX)(HANDLE, BOOL, PCONSOLE_FONT_INFOEX); -- cgit v0.10.2-6-g49f6 From 59de49f80d8f6067f19496750cc79632b2959ad9 Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Fri, 15 Jan 2016 14:24:29 +0100 Subject: mingw: avoid redefining S_* constants When compiling with MSys2's compiler, these constants are already defined. Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano diff --git a/compat/mingw.h b/compat/mingw.h index 10e2e4c..093a977 100644 --- a/compat/mingw.h +++ b/compat/mingw.h @@ -26,14 +26,18 @@ typedef int pid_t; #define S_ISLNK(x) (((x) & S_IFMT) == S_IFLNK) #define S_ISSOCK(x) 0 +#ifndef S_IRWXG #define S_IRGRP 0 #define S_IWGRP 0 #define S_IXGRP 0 #define S_IRWXG (S_IRGRP | S_IWGRP | S_IXGRP) +#endif +#ifndef S_IRWXO #define S_IROTH 0 #define S_IWOTH 0 #define S_IXOTH 0 #define S_IRWXO (S_IROTH | S_IWOTH | S_IXOTH) +#endif #define S_ISUID 0004000 #define S_ISGID 0002000 -- cgit v0.10.2-6-g49f6 From 7c00bc39eb39a26a9ad33de6e4a03fff45e86f5e Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Fri, 15 Jan 2016 14:24:34 +0100 Subject: mingw: avoid warnings when casting HANDLEs to int HANDLE is defined internally as a void *, but in many cases it is actually guaranteed to be a 32-bit integer. In these cases, GCC should not warn about a cast of a pointer to an integer of a different type because we know exactly what we are doing. Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano diff --git a/compat/mingw.c b/compat/mingw.c index 5edea29..7115e4e 100644 --- a/compat/mingw.c +++ b/compat/mingw.c @@ -6,6 +6,8 @@ #include "../run-command.h" #include "../cache.h" +#define HCAST(type, handle) ((type)(intptr_t)handle) + static const int delay[] = { 0, 1, 10, 20, 40 }; int err_win_to_posix(DWORD winerr) @@ -691,13 +693,13 @@ int pipe(int filedes[2]) errno = err_win_to_posix(GetLastError()); return -1; } - filedes[0] = _open_osfhandle((int)h[0], O_NOINHERIT); + filedes[0] = _open_osfhandle(HCAST(int, h[0]), O_NOINHERIT); if (filedes[0] < 0) { CloseHandle(h[0]); CloseHandle(h[1]); return -1; } - filedes[1] = _open_osfhandle((int)h[1], O_NOINHERIT); + filedes[1] = _open_osfhandle(HCAST(int, h[1]), O_NOINHERIT); if (filedes[1] < 0) { close(filedes[0]); CloseHandle(h[1]); @@ -1846,7 +1848,8 @@ void mingw_open_html(const char *unixpath) die("cannot run browser"); printf("Launching default browser to display HTML ...\n"); - r = (int)ShellExecute(NULL, "open", htmlpath, NULL, "\\", SW_SHOWNORMAL); + r = HCAST(int, ShellExecute(NULL, "open", htmlpath, + NULL, "\\", SW_SHOWNORMAL)); FreeLibrary(shell32); /* see the MSDN documentation referring to the result codes here */ if (r <= 32) { diff --git a/compat/poll/poll.c b/compat/poll/poll.c index db4e03e..b10adc7 100644 --- a/compat/poll/poll.c +++ b/compat/poll/poll.c @@ -76,7 +76,7 @@ #ifdef WIN32_NATIVE -#define IsConsoleHandle(h) (((long) (h) & 3) == 3) +#define IsConsoleHandle(h) (((long) (intptr_t) (h) & 3) == 3) static BOOL IsSocketHandle (HANDLE h) diff --git a/compat/winansi.c b/compat/winansi.c index 3aa5535..5dfa5ed 100644 --- a/compat/winansi.c +++ b/compat/winansi.c @@ -454,7 +454,8 @@ static HANDLE duplicate_handle(HANDLE hnd) HANDLE hresult, hproc = GetCurrentProcess(); if (!DuplicateHandle(hproc, hnd, hproc, &hresult, 0, TRUE, DUPLICATE_SAME_ACCESS)) - die_lasterr("DuplicateHandle(%li) failed", (long) hnd); + die_lasterr("DuplicateHandle(%li) failed", + (long) (intptr_t) hnd); return hresult; } -- cgit v0.10.2-6-g49f6 From 83c90da3c145c7f39cb321a3d3db59c167abd572 Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Fri, 15 Jan 2016 14:24:39 +0100 Subject: mingw: squash another warning about a cast MSys2's compiler is correct that casting a "void *" to a "DWORD" loses precision, but in the case of pthread_exit() we know that the value fits into a DWORD. Just like casting handles to DWORDs, let's work around this issue by casting to "intrptr_t" first, and immediately cast to the final type. Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano diff --git a/compat/win32/pthread.h b/compat/win32/pthread.h index 8ad1873..d3dd872 100644 --- a/compat/win32/pthread.h +++ b/compat/win32/pthread.h @@ -77,7 +77,7 @@ extern pthread_t pthread_self(void); static inline int pthread_exit(void *ret) { - ExitThread((DWORD)ret); + ExitThread((DWORD)(intptr_t)ret); } typedef DWORD pthread_key_t; -- cgit v0.10.2-6-g49f6 From 2921600afbcdd5f245475602cabba2fbaa5ff93f Mon Sep 17 00:00:00 2001 From: Johannes Schindelin Date: Fri, 15 Jan 2016 14:24:45 +0100 Subject: mingw: uglify (a, 0) definitions to shut up warnings When the result of a (a, 0) expression is not used, MSys2's GCC version finds it necessary to complain with a warning: right-hand operand of comma expression has no effect Let's just pretend to use the 0 value and have a peaceful and quiet life again. Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano diff --git a/compat/nedmalloc/malloc.c.h b/compat/nedmalloc/malloc.c.h index bf70037..b833ff9 100644 --- a/compat/nedmalloc/malloc.c.h +++ b/compat/nedmalloc/malloc.c.h @@ -1802,9 +1802,10 @@ struct win32_mlock_t volatile long threadid; }; +static inline int return_0(int i) { return 0; } #define MLOCK_T struct win32_mlock_t #define CURRENT_THREAD win32_getcurrentthreadid() -#define INITIAL_LOCK(sl) (memset(sl, 0, sizeof(MLOCK_T)), 0) +#define INITIAL_LOCK(sl) (memset(sl, 0, sizeof(MLOCK_T)), return_0(0)) #define ACQUIRE_LOCK(sl) win32_acquire_lock(sl) #define RELEASE_LOCK(sl) win32_release_lock(sl) #define TRY_LOCK(sl) win32_try_lock(sl) diff --git a/compat/win32/pthread.h b/compat/win32/pthread.h index d3dd872..20b35a2 100644 --- a/compat/win32/pthread.h +++ b/compat/win32/pthread.h @@ -18,7 +18,10 @@ */ #define pthread_mutex_t CRITICAL_SECTION -#define pthread_mutex_init(a,b) (InitializeCriticalSection((a)), 0) +static inline int return_0(int i) { + return 0; +} +#define pthread_mutex_init(a,b) return_0((InitializeCriticalSection((a)), 0)) #define pthread_mutex_destroy(a) DeleteCriticalSection((a)) #define pthread_mutex_lock EnterCriticalSection #define pthread_mutex_unlock LeaveCriticalSection -- cgit v0.10.2-6-g49f6