File 2881-erts-Add-support-for-static-Elixir-NIF-mod... of Package erlang

Overview Repositories Revisions Requests Users Attributes Meta

File 2881-erts-Add-support-for-static-Elixir-NIF-modules.patch of Package erlang (Revision c7414e24ac115dc7954a556db05458e9)

Currently displaying revision c7414e24ac115dc7954a556db05458e9 , Show latest

From 27775b501f272f69af2e40809a2c83a0bc25a722 Mon Sep 17 00:00:00 2001
From: Sverker Eriksson <sverker@erlang.org>
Date: Tue, 30 Nov 2021 18:10:56 +0100
Subject: [PATCH] erts: Add support for static Elixir NIF modules

or any modules with "exotic" characters in the name (like '.').

* Add STATIC_ERLANG_NIF_LIBNAME macro which is used both to
  identify the archive name and create a unique C identifier for
  the _nif_init function.

* Run all static *_nif_init functions unconditionally at VM boot to
  get the corresponding module names. erlang:load_nif/2 can then
  match only against module names and does not have to care about
  archive filenames for static nifs.
---
 HOWTO/INSTALL.md                    |  9 ++--
 erts/configure                      |  9 ++--
 erts/configure.ac                   |  2 +-
 erts/doc/src/erl_nif.xml            | 12 ++++--
 erts/emulator/beam/erl_nif.c        | 66 +++++++++++++++++------------
 erts/emulator/beam/erl_nif.h        | 19 ++++++++-
 erts/emulator/beam/global.h         | 17 ++++----
 erts/emulator/utils/make_driver_tab | 27 ++----------
 8 files changed, 88 insertions(+), 73 deletions(-)

diff --git a/HOWTO/INSTALL.md b/HOWTO/INSTALL.md
index dd06828e91..2b983497f0 100644
--- a/HOWTO/INSTALL.md
+++ b/HOWTO/INSTALL.md
@@ -396,11 +396,10 @@ Some of the available `configure` options are:
 that do not support dynamic linking of libraries it is possible to statically
 link nifs and drivers with the main Erlang VM binary. This is done by passing
 a comma separated list to the archives that you want to statically link. e.g.
- `--enable-static-nifs=/home/$USER/my_nif.a`. The path has to be absolute and the
- name of the archive has to be the same as the module, i.e. `my_nif` in the
- example above. This is also true for drivers, but then it is the driver name
- that has to be the same as the filename. You also have to define
- `STATIC_ERLANG_{NIF,DRIVER}` when compiling the .o files for the nif/driver.
+ `--enable-static-nifs=/home/$USER/my_nif.a`. The paths have to be absolute.
+ For drivers, the driver name has to be the same as the filename. You also
+ have to define `STATIC_ERLANG_NIF_LIBNAME` (see `erl_nif` documentation) or
+ `STATIC_ERLANG_DRIVER` when compiling the .o files for the nif/driver.
 If your nif/driver depends on some other dynamic library, you now have to link
 that to the Erlang VM binary. This is easily achieved by passing `LIBS=-llibname`
 to configure.
diff --git a/erts/configure b/erts/configure
index b97c6838d0..7d0aa2d1eb 100755
--- a/erts/configure
+++ b/erts/configure
@@ -1564,11 +1564,10 @@ Optional Features:
 into the main binary. It is also possible to give a
 list of nifs that should be linked statically. The
 list should be a comma separated and contain the
- absolute path to a .a archive for each nif that is
- to be statically linked. The name of the .a archive
- has to be the same as the name of the nif. Note that
- you have to link any external dependencies that the
- nifs have to the main binary, so for the crypto nif
+ absolute path to a .a archive for each nif lib
+ that is to be statically linked. Note that you have
+ to link any external dependencies, that the nifs have,
+ to the main binary. So for the crypto nifs
 you want to pass LIBS=-lcrypto to configure.
 --enable-static-drivers comma separated list of linked-in drivers to link
 statically with the main binary. The list should
diff --git a/erts/configure.in b/erts/configure.in
index 597ef38fee..e3b368e2b8 100644
--- a/erts/configure.in
+++ b/erts/configure.in
@@ -352,7 +352,7 @@ else
 fi
 
 AC_ARG_ENABLE(static-nifs,
-AS_HELP_STRING([--enable-static-nifs], [link nifs statically. If yes then all nifs in all Erlang/OTP applications will be statically linked into the main binary. It is also possible to give a list of nifs that should be linked statically. The list should be a comma separated and contain the absolute path to a .a archive for each nif that is to be statically linked. The name of the .a archive has to be the same as the name of the nif. Note that you have to link any external dependencies that the nifs have to the main binary, so for the crypto nif you want to pass LIBS=-lcrypto to configure.]),
+AS_HELP_STRING([--enable-static-nifs], [link nifs statically. If yes then all nifs in all Erlang/OTP applications will be statically linked into the main binary. It is also possible to give a list of nifs that should be linked statically. The list should be a comma separated and contain the absolute path to a .a archive for each nif that is to be statically linked. Note that you have to link any external dependencies, that the nifs have, to the main binary. So for the crypto nifs you want to pass LIBS=-lcrypto to configure.]),
 	 STATIC_NIFS="$enableval",
 	 STATIC_NIFS=no)
 AC_SUBST(STATIC_NIFS)
diff --git a/erts/doc/src/erl_nif.xml b/erts/doc/src/erl_nif.xml
index a8c21f8155..32c81ba97e 100644
--- a/erts/doc/src/erl_nif.xml
+++ b/erts/doc/src/erl_nif.xml
@@ -599,9 +599,15 @@ int writeiovec(ErlNifEnv *env, ERL_NIF_TERM term, ERL_NIF_TERM *tail,
 The fourth argument <c>NULL</c> is ignored. It
 was earlier used for the deprecated <c>reload</c> callback
 which is no longer supported since OTP 20.
- If compiling a NIF for static inclusion through
- <c>--enable-static-nifs</c>, you must define <c>STATIC_ERLANG_NIF</c>
- before the <c>ERL_NIF_INIT</c> declaration.
+ If compiling a NIF lib for static inclusion through
+ <c>--enable-static-nifs</c>, then the macro
+ <c>STATIC_ERLANG_NIF_LIBNAME</c> must be defined as the name of the
+ archive file (excluding file extension .a) without string
+ quotations. It must only contain characters allowed in a C
+ indentifier. The macro must be defined before <c>erl_nif.h</c> is
+ included. If the older macro <c>STATIC_ERLANG_NIF</c> is instead
+ used, then the name of the archive file must match the name of the
+ module.
 </item>
 <tag><marker id="load"/><c>int (*load)(ErlNifEnv* caller_env, void** priv_data,
 ERL_NIF_TERM load_info)</c></tag>
diff --git a/erts/emulator/beam/erl_nif.c b/erts/emulator/beam/erl_nif.c
index 9dd484202f..1c95d920f4 100644
--- a/erts/emulator/beam/erl_nif.c
+++ b/erts/emulator/beam/erl_nif.c
@@ -88,7 +88,7 @@ struct erl_module_nif {
 ErtsThrPrgrLaterOp lop;
 
 void* priv_data;
- void* handle; /* "dlopen" */
+ void* handle; /* "dlopen", NULL for static linked */
 struct enif_entry_t entry;
 erts_refc_t dynlib_refc; /* References to loaded native code
 +1 erl_module_instance
@@ -2268,6 +2268,8 @@ static void deref_nifmod(struct erl_module_nif* lib)
 }
 }
 
+static ErtsStaticNif* is_static_nif_module(Eterm mod_atom);
+
 static void close_dynlib(struct erl_module_nif* lib)
 {
 ASSERT(lib != NULL);
@@ -2281,10 +2283,11 @@ static void close_dynlib(struct erl_module_nif* lib)
 	lib->entry.unload(&msg_env.env, lib->priv_data);
 post_nif_noproc(&msg_env);
 }
- if (!erts_is_static_nif(lib->handle))
+ if (lib->handle) {
 erts_sys_ddll_close(lib->handle);
+ lib->handle = NULL;
+ }
 
- lib->handle = NULL;
 deref_nifmod(lib);
 }
 
@@ -4403,13 +4406,12 @@ Eterm erts_load_nif(Process *c_p, ErtsCodePtr I, Eterm filename, Eterm args)
 int i, err, encoding;
 Module* module_p;
 Eterm mod_atom;
- const Atom* mod_atomp;
 Eterm f_atom;
 const ErtsCodeMFA* caller;
 ErtsSysDdllError errdesc = ERTS_SYS_DDLL_ERROR_INIT;
 Eterm ret = am_ok;
 int veto;
- int taint = 1;
+ int is_static = 0;
 struct erl_module_nif* lib = NULL;
 struct erl_module_instance* this_mi;
 struct erl_module_instance* prev_mi;
@@ -4435,24 +4437,11 @@ Eterm erts_load_nif(Process *c_p, ErtsCodePtr I, Eterm filename, Eterm args)
 module_p = erts_get_module(mod_atom, erts_active_code_ix());
 ASSERT(module_p != NULL);
 
- mod_atomp = atom_tab(atom_val(mod_atom));
 {
- ErtsStaticNifEntry* sne;
- sne = erts_static_nif_get_nif_init((char*)mod_atomp->name, mod_atomp->len);
- if (sne == NULL) {
- /* Second lookup in the static nif table based on the filename */
- /* this allows Elixir modules which always have dots '.' in their */
- /* module names to be loaded */
- char *basename = strrchr(lib_name, '/');
- if (basename) {
- basename++;
- sne = erts_static_nif_get_nif_init(basename, strlen(basename));
- }
- } 
+ ErtsStaticNif* sne = is_static_nif_module(mod_atom);
 if (sne != NULL) {
- init_func = sne->nif_init;
- handle = init_func;
- taint = sne->taint;
+ entry = sne->entry;
+ is_static = 1;
 }
 }
 this_mi = &module_p->curr;
@@ -4475,7 +4464,7 @@ Eterm erts_load_nif(Process *c_p, ErtsCodePtr I, Eterm filename, Eterm args)
 ret = load_nif_error(c_p,"reload","NIF library already loaded"
 " (reload disallowed since OTP 20).");
 }
- else if (init_func == NULL &&
+ else if (!is_static &&
 (err=erts_sys_ddll_open(lib_name, &handle, &errdesc)) != ERL_DE_NO_ERROR) {
 	const char slogan[] = "Failed to load NIF library";
 	if (strstr(errdesc.str, lib_name) != NULL) {
@@ -4485,14 +4474,15 @@ Eterm erts_load_nif(Process *c_p, ErtsCodePtr I, Eterm filename, Eterm args)
 	 ret = load_nif_error(c_p, "load_failed", "%s %s: '%s'", slogan, lib_name, errdesc.str);
 	}
 }
- else if (init_func == NULL &&
+ else if (!is_static &&
 	 erts_sys_ddll_load_nif_init(handle, &init_func, &errdesc) != ERL_DE_NO_ERROR) {
 	ret = load_nif_error(c_p, bad_lib, "Failed to find library init"
 			 " function: '%s'", errdesc.str);
 	
 }
- else if ((taint ? erts_add_taint(mod_atom) : 0,
-	 (entry = erts_sys_ddll_call_nif_init(init_func)) == NULL)) {
+ else if (!is_static &&
+ (erts_add_taint(mod_atom),
+ (entry = erts_sys_ddll_call_nif_init(init_func)) == NULL)) {
 	ret = load_nif_error(c_p, bad_lib, "Library init-call unsuccessful");
 }
 else if (entry->major > ERL_NIF_MAJOR_VERSION
@@ -4711,7 +4701,7 @@ Eterm erts_load_nif(Process *c_p, ErtsCodePtr I, Eterm filename, Eterm args)
 }
 	 erts_free(ERTS_ALC_T_NIF, lib);
 	}
-	if (handle != NULL && !erts_is_static_nif(handle)) {
+	if (handle != NULL) {
 	 erts_sys_ddll_close(handle);
 	}
 	erts_sys_ddll_free_error(&errdesc);
@@ -4965,6 +4955,29 @@ static void erase_hashed_stubs(ErtsNifFinish* fin)
 erts_rwmtx_rwunlock(&erts_nif_call_tab_lock);
 }
 
+static void static_nifs_init(void)
+{
+ ErtsStaticNif* p;
+
+ for (p = erts_static_nif_tab; p->nif_init != NULL; p++) {
+ ASSERT(p->entry == NULL && p->mod_atom == THE_NON_VALUE);
+ p->entry = erts_sys_ddll_call_nif_init(p->nif_init);
+ p->mod_atom = mkatom(p->entry->name);
+ if (p->taint)
+ erts_add_taint(p->mod_atom);
+ }
+}
+
+static ErtsStaticNif* is_static_nif_module(Eterm mod_atom)
+{
+ ErtsStaticNif* p;
+ for (p = erts_static_nif_tab; p->nif_init != NULL; p++)
+ if (mod_atom == p->mod_atom)
+ return p;
+ return NULL;
+}
+
+
 void
 erts_unload_nif(struct erl_module_nif* lib)
 {
@@ -5019,6 +5032,7 @@ void erl_nif_init()
 resource_type_list.name = THE_NON_VALUE;
 
 nif_call_table_init();
+ static_nifs_init();
 }
 
 int erts_nif_get_funcs(struct erl_module_nif* mod,
diff --git a/erts/emulator/beam/erl_nif.h b/erts/emulator/beam/erl_nif.h
index 42c4340668..28ea6c764e 100644
--- a/erts/emulator/beam/erl_nif.h
+++ b/erts/emulator/beam/erl_nif.h
@@ -335,6 +335,10 @@ extern TWinDynNifCallbacks WinDynNifCallbacks;
 # undef ERL_NIF_API_FUNC_DECL
 #endif
 
+#ifdef STATIC_ERLANG_NIF_LIBNAME
+# define STATIC_ERLANG_NIF
+#endif
+
 #if (defined(__WIN32__) || defined(_WIN32) || defined(_WIN32_)) && !defined(STATIC_ERLANG_DRIVER) && !defined(STATIC_ERLANG_NIF)
 # define ERL_NIF_API_FUNC_MACRO(NAME) (WinDynNifCallbacks.NAME)
 # include "erl_nif_api_funcs.h"
@@ -365,12 +369,23 @@ extern TWinDynNifCallbacks WinDynNifCallbacks;
 # endif
 #endif
 
+
 #ifdef STATIC_ERLANG_NIF
-# define ERL_NIF_INIT_DECL(MODNAME) ErlNifEntry* MODNAME ## _nif_init(ERL_NIF_INIT_ARGS)
+# ifdef STATIC_ERLANG_NIF_LIBNAME
+# define ERL_NIF_INIT_NAME(MODNAME) ERL_NIF_INIT_NAME2(STATIC_ERLANG_NIF_LIBNAME)
+# define ERL_NIF_INIT_NAME2(LIB) ERL_NIF_INIT_NAME3(LIB)
+# define ERL_NIF_INIT_NAME3(LIB) LIB ## _nif_init
+# else
+# define ERL_NIF_INIT_NAME(MODNAME) MODNAME ## _nif_init
+# endif
+# define ERL_NIF_INIT_DECL(MODNAME) \
+ ErlNifEntry* ERL_NIF_INIT_NAME(MODNAME)(ERL_NIF_INIT_ARGS)
 #else
-# define ERL_NIF_INIT_DECL(MODNAME) ERL_NIF_INIT_EXPORT ErlNifEntry* nif_init(ERL_NIF_INIT_ARGS)
+# define ERL_NIF_INIT_DECL(MODNAME) \
+ ERL_NIF_INIT_EXPORT ErlNifEntry* nif_init(ERL_NIF_INIT_ARGS)
 #endif
 
+
 #ifdef __cplusplus
 }
 # define ERL_NIF_INIT_PROLOGUE extern "C" {
diff --git a/erts/emulator/beam/global.h b/erts/emulator/beam/global.h
index b0dd2fe57c..9a3c112f5a 100644
--- a/erts/emulator/beam/global.h
+++ b/erts/emulator/beam/global.h
@@ -1308,14 +1308,15 @@ typedef struct {
 ErlDrvEntry* de;
 int taint;
 } ErtsStaticDriver;
-typedef void *(*ErtsStaticNifInitFPtr)(void);
-typedef struct ErtsStaticNifEntry_ {
- const char *nif_name;
- ErtsStaticNifInitFPtr nif_init;
- int taint;
-} ErtsStaticNifEntry;
-ErtsStaticNifEntry* erts_static_nif_get_nif_init(const char *name, int len);
-int erts_is_static_nif(void *handle);
+typedef void* ErtsStaticNifInitF(void);
+typedef struct {
+ ErtsStaticNifInitF* const nif_init;
+ const int taint;
+
+ Eterm mod_atom;
+ ErlNifEntry* entry;
+} ErtsStaticNif;
+extern ErtsStaticNif erts_static_nif_tab[];
 void erts_init_static_drivers(void);
 
 /* erl_drv_thread.c */
diff --git a/erts/emulator/utils/make_driver_tab b/erts/emulator/utils/make_driver_tab
index 78b6fba254..0dab24b795 100755
--- a/erts/emulator/utils/make_driver_tab
+++ b/erts/emulator/utils/make_driver_tab
@@ -130,38 +130,19 @@ foreach (@static_nifs) {
 }
 
 # The array itself
-print "static ErtsStaticNifEntry static_nif_tab[] =\n{\n";
+print "ErtsStaticNif erts_static_nif_tab[] =\n{\n";
 
 foreach (@emu_nifs) {
 my $d = ${_};
 $d =~ s/\.debug//; # strip .debug
- print " {\"${_}\", &".$d."_nif_init, 0},\n";
+ print " {&".$d."_nif_init, 0, THE_NON_VALUE, NULL},\n";
 }
 foreach (@static_nifs) {
 my $d = ${_};
 $d =~ s/\.debug//; # strip .debug
- print " {\"${_}\", &".$d."_nif_init, 1},\n";
+ print " {&".$d."_nif_init, 1, THE_NON_VALUE, NULL},\n";
 }
 
-print " {NULL,NULL}\n};\n";
-
-print <<EOF;
-ErtsStaticNifEntry* erts_static_nif_get_nif_init(const char *name, int len) {
- ErtsStaticNifEntry* p;
- for (p = static_nif_tab; p->nif_name != NULL; p++)
- if (strncmp(p->nif_name, name, len) == 0 && p->nif_name[len] == 0)
- return p;
- return NULL;
-}
-
-int erts_is_static_nif(void *handle) {
- ErtsStaticNifEntry* p;
- for (p = static_nif_tab; p->nif_name != NULL; p++)
- if (((void*)p->nif_init) == handle)
- return 1;
- return 0;
-}
-
-EOF
+print " {NULL}\n};\n";
 
 # That's it
-- 
2.31.1

Places

File 2881-erts-Add-support-for-static-Elixir-NIF-modules.patch of Package erlang (Revision c7414e24ac115dc7954a556db05458e9)

Places