EVOLUTION-MANAGER
Edit File: embedding.html
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <title>Using CFFI for embedding — CFFI 1.9.1 documentation</title> <link rel="stylesheet" href="_static/default.css" type="text/css" /> <link rel="stylesheet" href="_static/pygments.css" type="text/css" /> <script type="text/javascript"> var DOCUMENTATION_OPTIONS = { URL_ROOT: '', VERSION: '1.9.1', COLLAPSE_INDEX: false, FILE_SUFFIX: '.html', HAS_SOURCE: true }; </script> <script type="text/javascript" src="_static/jquery.js"></script> <script type="text/javascript" src="_static/underscore.js"></script> <script type="text/javascript" src="_static/doctools.js"></script> <link rel="top" title="CFFI 1.9.1 documentation" href="index.html" /> <link rel="prev" title="Preparing and Distributing modules" href="cdef.html" /> </head> <body> <div class="related"> <h3>Navigation</h3> <ul> <li class="right" style="margin-right: 10px"> <a href="genindex.html" title="General Index" accesskey="I">index</a></li> <li class="right" > <a href="cdef.html" title="Preparing and Distributing modules" accesskey="P">previous</a> |</li> <li><a href="index.html">CFFI 1.9.1 documentation</a> »</li> </ul> </div> <div class="document"> <div class="documentwrapper"> <div class="bodywrapper"> <div class="body"> <div class="section" id="using-cffi-for-embedding"> <h1><a class="toc-backref" href="#id7">Using CFFI for embedding</a><a class="headerlink" href="#using-cffi-for-embedding" title="Permalink to this headline">¶</a></h1> <div class="contents topic" id="contents"> <p class="topic-title first">Contents</p> <ul class="simple"> <li><a class="reference internal" href="#using-cffi-for-embedding" id="id7">Using CFFI for embedding</a><ul> <li><a class="reference internal" href="#usage" id="id8">Usage</a></li> <li><a class="reference internal" href="#more-reading" id="id9">More reading</a></li> <li><a class="reference internal" href="#troubleshooting" id="id10">Troubleshooting</a></li> <li><a class="reference internal" href="#issues-about-using-the-so" id="id11">Issues about using the .so</a></li> <li><a class="reference internal" href="#using-multiple-cffi-made-dlls" id="id12">Using multiple CFFI-made DLLs</a></li> <li><a class="reference internal" href="#multithreading" id="id13">Multithreading</a></li> <li><a class="reference internal" href="#testing" id="id14">Testing</a></li> <li><a class="reference internal" href="#embedding-and-extending" id="id15">Embedding and Extending</a></li> </ul> </li> </ul> </div> <p>You can use CFFI to generate a <tt class="docutils literal"><span class="pre">.so/.dll/.dylib</span></tt> which exports the API of your choice to any C application that wants to link with this <tt class="docutils literal"><span class="pre">.so/.dll/.dylib</span></tt>.</p> <p>The general idea is as follows:</p> <ul class="simple"> <li>You write and execute a Python script, which produces a <tt class="docutils literal"><span class="pre">.so/.dll/.dylib</span></tt> file with the API of your choice. The script also gives some Python code to be “frozen” inside the <tt class="docutils literal"><span class="pre">.so</span></tt>.</li> <li>At runtime, the C application loads this <tt class="docutils literal"><span class="pre">.so/.dll/.dylib</span></tt> without having to know that it was produced by Python and CFFI.</li> <li>The first time a C function is called, Python is initialized and the frozen Python code is executed.</li> <li>The frozen Python code attaches Python functions that implement the C functions of your API, which are then used for all subsequent C function calls.</li> </ul> <p>One of the goals of this approach is to be entirely independent from the CPython C API: no <tt class="docutils literal"><span class="pre">Py_Initialize()</span></tt> nor <tt class="docutils literal"><span class="pre">PyRun_SimpleString()</span></tt> nor even <tt class="docutils literal"><span class="pre">PyObject</span></tt>. It works identically on CPython and PyPy.</p> <p>This is entirely <em>new in version 1.5.</em> (PyPy contains CFFI 1.5 since release 5.0.)</p> <div class="section" id="usage"> <h2><a class="toc-backref" href="#id8">Usage</a><a class="headerlink" href="#usage" title="Permalink to this headline">¶</a></h2> <p>See the <a class="reference external" href="overview.html#embedding">paragraph in the overview page</a> for a quick introduction. In this section, we explain every step in more details. We will use here this slightly expanded example:</p> <div class="highlight-c"><div class="highlight"><pre><span class="cm">/* file plugin.h */</span> <span class="k">typedef</span> <span class="k">struct</span> <span class="p">{</span> <span class="kt">int</span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">;</span> <span class="p">}</span> <span class="n">point_t</span><span class="p">;</span> <span class="k">extern</span> <span class="kt">int</span> <span class="n">do_stuff</span><span class="p">(</span><span class="n">point_t</span> <span class="o">*</span><span class="p">);</span> </pre></div> </div> <div class="highlight-python"><div class="highlight"><pre><span class="c"># file plugin_build.py</span> <span class="kn">import</span> <span class="nn">cffi</span> <span class="n">ffibuilder</span> <span class="o">=</span> <span class="n">cffi</span><span class="o">.</span><span class="n">FFI</span><span class="p">()</span> <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s">'plugin.h'</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span> <span class="n">ffibuilder</span><span class="o">.</span><span class="n">embedding_api</span><span class="p">(</span><span class="n">f</span><span class="o">.</span><span class="n">read</span><span class="p">())</span> <span class="n">ffibuilder</span><span class="o">.</span><span class="n">set_source</span><span class="p">(</span><span class="s">"my_plugin"</span><span class="p">,</span> <span class="s">r'''</span> <span class="s"> #include "plugin.h"</span> <span class="s">'''</span><span class="p">)</span> <span class="n">ffibuilder</span><span class="o">.</span><span class="n">embedding_init_code</span><span class="p">(</span><span class="s">"""</span> <span class="s"> from my_plugin import ffi</span> <span class="s"> @ffi.def_extern()</span> <span class="s"> def do_stuff(p):</span> <span class="s"> print("adding </span><span class="si">%d</span><span class="s"> and </span><span class="si">%d</span><span class="s">" % (p.x, p.y))</span> <span class="s"> return p.x + p.y</span> <span class="s">"""</span><span class="p">)</span> <span class="n">ffibuilder</span><span class="o">.</span><span class="n">compile</span><span class="p">(</span><span class="n">target</span><span class="o">=</span><span class="s">"plugin-1.5.*"</span><span class="p">,</span> <span class="n">verbose</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span> </pre></div> </div> <p>Running the code above produces a <em>DLL</em>, i,e, a dynamically-loadable library. It is a file with the extension <tt class="docutils literal"><span class="pre">.dll</span></tt> on Windows, <tt class="docutils literal"><span class="pre">.dylib</span></tt> on Mac OS/X, or <tt class="docutils literal"><span class="pre">.so</span></tt> on other platforms. As usual, it is produced by generating some intermediate <tt class="docutils literal"><span class="pre">.c</span></tt> code and then calling the regular platform-specific C compiler. See <a class="reference internal" href="#issues-about-using-the-so">below</a> for some pointers to C-level issues with using the produced library.</p> <p>Here are some details about the methods used above:</p> <ul> <li><p class="first"><strong>ffibuilder.embedding_api(source):</strong> parses the given C source, which declares functions that you want to be exported by the DLL. It can also declare types, constants and global variables that are part of the C-level API of your DLL.</p> <p>The functions that are found in <tt class="docutils literal"><span class="pre">source</span></tt> will be automatically defined in the <tt class="docutils literal"><span class="pre">.c</span></tt> file: they will contain code that initializes the Python interpreter the first time any of them is called, followed by code to call the attached Python function (with <tt class="docutils literal"><span class="pre">@ffi.def_extern()</span></tt>, see next point).</p> <p>The global variables, on the other hand, are not automatically produced. You have to write their definition explicitly in <tt class="docutils literal"><span class="pre">ffibuilder.set_source()</span></tt>, as regular C code (see the point after next).</p> </li> <li><p class="first"><strong>ffibuilder.embedding_init_code(python_code):</strong> this gives initialization-time Python source code. This code is copied (“frozen”) inside the DLL. At runtime, the code is executed when the DLL is first initialized, just after Python itself is initialized. This newly initialized Python interpreter has got an extra “built-in” module that can be loaded magically without accessing any files, with a line like “<tt class="docutils literal"><span class="pre">from</span> <span class="pre">my_plugin</span> <span class="pre">import</span> <span class="pre">ffi,</span> <span class="pre">lib</span></tt>”. The name <tt class="docutils literal"><span class="pre">my_plugin</span></tt> comes from the first argument to <tt class="docutils literal"><span class="pre">ffibuilder.set_source()</span></tt>. This module represents “the caller’s C world” from the point of view of Python.</p> <p>The initialization-time Python code can import other modules or packages as usual. You may have typical Python issues like needing to set up <tt class="docutils literal"><span class="pre">sys.path</span></tt> somehow manually first.</p> <p>For every function declared within <tt class="docutils literal"><span class="pre">ffibuilder.embedding_api()</span></tt>, the initialization-time Python code or one of the modules it imports should use the decorator <tt class="docutils literal"><span class="pre">@ffi.def_extern()</span></tt> to attach a corresponding Python function to it.</p> <p>If the initialization-time Python code fails with an exception, then you get a traceback printed to stderr, along with more information to help you identify problems like wrong <tt class="docutils literal"><span class="pre">sys.path</span></tt>. If some function remains unattached at the time where the C code tries to call it, an error message is also printed to stderr and the function returns zero/null.</p> <p>Note that the CFFI module never calls <tt class="docutils literal"><span class="pre">exit()</span></tt>, but CPython itself contains code that calls <tt class="docutils literal"><span class="pre">exit()</span></tt>, for example if importing <tt class="docutils literal"><span class="pre">site</span></tt> fails. This may be worked around in the future.</p> </li> <li><p class="first"><strong>ffibuilder.set_source(c_module_name, c_code):</strong> set the name of the module from Python’s point of view. It also gives more C code which will be included in the generated C code. In trivial examples it can be an empty string. It is where you would <tt class="docutils literal"><span class="pre">#include</span></tt> some other files, define global variables, and so on. The macro <tt class="docutils literal"><span class="pre">CFFI_DLLEXPORT</span></tt> is available to this C code: it expands to the platform-specific way of saying “the following declaration should be exported from the DLL”. For example, you would put “<tt class="docutils literal"><span class="pre">extern</span> <span class="pre">int</span> <span class="pre">my_glob;</span></tt>” in <tt class="docutils literal"><span class="pre">ffibuilder.embedding_api()</span></tt> and “<tt class="docutils literal"><span class="pre">CFFI_DLLEXPORT</span> <span class="pre">int</span> <span class="pre">my_glob</span> <span class="pre">=</span> <span class="pre">42;</span></tt>” in <tt class="docutils literal"><span class="pre">ffibuilder.set_source()</span></tt>.</p> <p>Currently, any <em>type</em> declared in <tt class="docutils literal"><span class="pre">ffibuilder.embedding_api()</span></tt> must also be present in the <tt class="docutils literal"><span class="pre">c_code</span></tt>. This is automatic if this code contains a line like <tt class="docutils literal"><span class="pre">#include</span> <span class="pre">"plugin.h"</span></tt> in the example above.</p> </li> <li><p class="first"><strong>ffibuilder.compile([target=...] [, verbose=True]):</strong> make the C code and compile it. By default, it produces a file called <tt class="docutils literal"><span class="pre">c_module_name.dll</span></tt>, <tt class="docutils literal"><span class="pre">c_module_name.dylib</span></tt> or <tt class="docutils literal"><span class="pre">c_module_name.so</span></tt>, but the default can be changed with the optional <tt class="docutils literal"><span class="pre">target</span></tt> keyword argument. You can use <tt class="docutils literal"><span class="pre">target="foo.*"</span></tt> with a literal <tt class="docutils literal"><span class="pre">*</span></tt> to ask for a file called <tt class="docutils literal"><span class="pre">foo.dll</span></tt> on Windows, <tt class="docutils literal"><span class="pre">foo.dylib</span></tt> on OS/X and <tt class="docutils literal"><span class="pre">foo.so</span></tt> elsewhere. One reason for specifying an alternate <tt class="docutils literal"><span class="pre">target</span></tt> is to include characters not usually allowed in Python module names, like “<tt class="docutils literal"><span class="pre">plugin-1.5.*</span></tt>”.</p> <p>For more complicated cases, you can call instead <tt class="docutils literal"><span class="pre">ffibuilder.emit_c_code("foo.c")</span></tt> and compile the resulting <tt class="docutils literal"><span class="pre">foo.c</span></tt> file using other means. CFFI’s compilation logic is based on the standard library <tt class="docutils literal"><span class="pre">distutils</span></tt> package, which is really developed and tested for the purpose of making CPython extension modules, not other DLLs.</p> </li> </ul> </div> <div class="section" id="more-reading"> <h2><a class="toc-backref" href="#id9">More reading</a><a class="headerlink" href="#more-reading" title="Permalink to this headline">¶</a></h2> <p>If you’re reading this page about embedding and you are not familiar with CFFI already, here are a few pointers to what you could read next:</p> <ul> <li><p class="first">For the <tt class="docutils literal"><span class="pre">@ffi.def_extern()</span></tt> functions, integer C types are passed simply as Python integers; and simple pointers-to-struct and basic arrays are all straightforward enough. However, sooner or later you will need to read about this topic in more details <a class="reference external" href="using.html#working">here</a>.</p> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">@ffi.def_extern()</span></tt>: see <a class="reference external" href="using.html#def-extern">documentation here,</a> notably on what happens if the Python function raises an exception.</p> </li> <li><p class="first">To create Python objects attached to C data, one common solution is to use <tt class="docutils literal"><span class="pre">ffi.new_handle()</span></tt>. See documentation <a class="reference external" href="ref.html#ffi-new-handle">here</a>.</p> </li> <li><p class="first">In embedding mode, the major direction is C code that calls Python functions. This is the opposite of the regular extending mode of CFFI, in which the major direction is Python code calling C. That’s why the page <a class="reference external" href="using.html">Using the ffi/lib objects</a> talks first about the latter, and why the direction “C code that calls Python” is generally referred to as “callbacks” in that page. If you also need to have your Python code call C code, read more about <a class="reference internal" href="#embedding-and-extending">Embedding and Extending</a> below.</p> </li> <li><p class="first"><tt class="docutils literal"><span class="pre">ffibuilder.embedding_api(source)</span></tt>: follows the same syntax as <tt class="docutils literal"><span class="pre">ffibuilder.cdef()</span></tt>, <a class="reference external" href="cdef.html#cdef">documented here.</a> You can use the “<tt class="docutils literal"><span class="pre">...</span></tt>” syntax as well, although in practice it may be less useful than it is for <tt class="docutils literal"><span class="pre">cdef()</span></tt>. On the other hand, it is expected that often the C sources that you need to give to <tt class="docutils literal"><span class="pre">ffibuilder.embedding_api()</span></tt> would be exactly the same as the content of some <tt class="docutils literal"><span class="pre">.h</span></tt> file that you want to give to users of your DLL. That’s why the example above does this:</p> <div class="highlight-python"><div class="highlight"><pre><span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s">'foo.h'</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span> <span class="n">ffibuilder</span><span class="o">.</span><span class="n">embedding_api</span><span class="p">(</span><span class="n">f</span><span class="o">.</span><span class="n">read</span><span class="p">())</span> </pre></div> </div> <p>Note that a drawback of this approach is that <tt class="docutils literal"><span class="pre">ffibuilder.embedding_api()</span></tt> doesn’t support <tt class="docutils literal"><span class="pre">#ifdef</span></tt> directives. You may have to use a more convoluted expression like:</p> <div class="highlight-python"><div class="highlight"><pre><span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s">'foo.h'</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span> <span class="n">lines</span> <span class="o">=</span> <span class="p">[</span><span class="n">line</span> <span class="k">for</span> <span class="n">line</span> <span class="ow">in</span> <span class="n">f</span> <span class="k">if</span> <span class="ow">not</span> <span class="n">line</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s">'#'</span><span class="p">)]</span> <span class="n">ffibuilder</span><span class="o">.</span><span class="n">embedding_api</span><span class="p">(</span><span class="s">''</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">lines</span><span class="p">))</span> </pre></div> </div> <p>As in the example above, you can also use the same <tt class="docutils literal"><span class="pre">foo.h</span></tt> from <tt class="docutils literal"><span class="pre">ffibuilder.set_source()</span></tt>:</p> <div class="highlight-python"><div class="highlight"><pre><span class="n">ffibuilder</span><span class="o">.</span><span class="n">set_source</span><span class="p">(</span><span class="s">'module_name'</span><span class="p">,</span> <span class="s">r'''</span> <span class="s"> #include "foo.h"</span> <span class="s">'''</span><span class="p">)</span> </pre></div> </div> </li> </ul> </div> <div class="section" id="troubleshooting"> <h2><a class="toc-backref" href="#id10">Troubleshooting</a><a class="headerlink" href="#troubleshooting" title="Permalink to this headline">¶</a></h2> <p>The error message</p> <blockquote> <div>cffi extension module ‘c_module_name’ has unknown version 0x2701</div></blockquote> <p>means that the running Python interpreter located a CFFI version older than 1.5. CFFI 1.5 or newer must be installed in the running Python.</p> </div> <div class="section" id="issues-about-using-the-so"> <h2><a class="toc-backref" href="#id11">Issues about using the .so</a><a class="headerlink" href="#issues-about-using-the-so" title="Permalink to this headline">¶</a></h2> <p>This paragraph describes issues that are not necessarily specific to CFFI. It assumes that you have obtained the <tt class="docutils literal"><span class="pre">.so/.dylib/.dll</span></tt> file as described above, but that you have troubles using it. (In summary: it is a mess. This is my own experience, slowly built by using Google and by listening to reports from various platforms. Please report any inaccuracies in this paragraph or better ways to do things.)</p> <ul> <li><p class="first">The file produced by CFFI should follow this naming pattern: <tt class="docutils literal"><span class="pre">libmy_plugin.so</span></tt> on Linux, <tt class="docutils literal"><span class="pre">libmy_plugin.dylib</span></tt> on Mac, or <tt class="docutils literal"><span class="pre">my_plugin.dll</span></tt> on Windows (no <tt class="docutils literal"><span class="pre">lib</span></tt> prefix on Windows).</p> </li> <li><p class="first">First note that this file does not contain the Python interpreter nor the standard library of Python. You still need it to be somewhere. There are ways to compact it to a smaller number of files, but this is outside the scope of CFFI (please report if you used some of these ways successfully so that I can add some links here).</p> </li> <li><p class="first">In what we’ll call the “main program”, the <tt class="docutils literal"><span class="pre">.so</span></tt> can be either used dynamically (e.g. by calling <tt class="docutils literal"><span class="pre">dlopen()</span></tt> or <tt class="docutils literal"><span class="pre">LoadLibrary()</span></tt> inside the main program), or at compile-time (e.g. by compiling it with <tt class="docutils literal"><span class="pre">gcc</span> <span class="pre">-lmy_plugin</span></tt>). The former case is always used if you’re building a plugin for a program, and the program itself doesn’t need to be recompiled. The latter case is for making a CFFI library that is more tightly integrated inside the main program.</p> </li> <li><p class="first">In the case of compile-time usage: you can add the gcc option <tt class="docutils literal"><span class="pre">-Lsome/path/</span></tt> before <tt class="docutils literal"><span class="pre">-lmy_plugin</span></tt> to describe where the <tt class="docutils literal"><span class="pre">libmy_plugin.so</span></tt> is. On some platforms, notably Linux, <tt class="docutils literal"><span class="pre">gcc</span></tt> will complain if it can find <tt class="docutils literal"><span class="pre">libmy_plugin.so</span></tt> but not <tt class="docutils literal"><span class="pre">libpython27.so</span></tt> or <tt class="docutils literal"><span class="pre">libpypy-c.so</span></tt>. To fix it, you need to call <tt class="docutils literal"><span class="pre">LD_LIBRARY_PATH=/some/path/to/libpypy</span> <span class="pre">gcc</span></tt>.</p> </li> <li><p class="first">When actually executing the main program, it needs to find the <tt class="docutils literal"><span class="pre">libmy_plugin.so</span></tt> but also <tt class="docutils literal"><span class="pre">libpython27.so</span></tt> or <tt class="docutils literal"><span class="pre">libpypy-c.so</span></tt>. For PyPy, unpack a PyPy distribution and you get a full directory structure with <tt class="docutils literal"><span class="pre">libpypy-c.so</span></tt> inside a <tt class="docutils literal"><span class="pre">bin</span></tt> subdirectory, or on Windows <tt class="docutils literal"><span class="pre">pypy-c.dll</span></tt> inside the top directory; you must not move this file around, but just point to it. One way to point to it is by running the main program with some environment variable: <tt class="docutils literal"><span class="pre">LD_LIBRARY_PATH=/some/path/to/libpypy</span></tt> on Linux, <tt class="docutils literal"><span class="pre">DYLD_LIBRARY_PATH=/some/path/to/libpypy</span></tt> on OS/X.</p> </li> <li><p class="first">You can avoid the <tt class="docutils literal"><span class="pre">LD_LIBRARY_PATH</span></tt> issue if you compile <tt class="docutils literal"><span class="pre">libmy_plugin.so</span></tt> with the path hard-coded inside in the first place. On Linux, this is done by <tt class="docutils literal"><span class="pre">gcc</span> <span class="pre">-Wl,-rpath=/some/path</span></tt>. You would put this option in <tt class="docutils literal"><span class="pre">ffibuilder.set_source("my_plugin",</span> <span class="pre">...,</span> <span class="pre">extra_link_args=['-Wl,-rpath=/some/path/to/libpypy'])</span></tt>. The path can start with <tt class="docutils literal"><span class="pre">$ORIGIN</span></tt> to mean “the directory where <tt class="docutils literal"><span class="pre">libmy_plugin.so</span></tt> is”. You can then specify a path relative to that place, like <tt class="docutils literal"><span class="pre">extra_link_args=['-Wl,-rpath=$ORIGIN/../venv/bin']</span></tt>. Use <tt class="docutils literal"><span class="pre">ldd</span> <span class="pre">libmy_plugin.so</span></tt> to look at what path is currently compiled in after the expansion of <tt class="docutils literal"><span class="pre">$ORIGIN</span></tt>.)</p> <p>After this, you don’t need <tt class="docutils literal"><span class="pre">LD_LIBRARY_PATH</span></tt> any more to locate <tt class="docutils literal"><span class="pre">libpython27.so</span></tt> or <tt class="docutils literal"><span class="pre">libpypy-c.so</span></tt> at runtime. In theory it should also cover the call to <tt class="docutils literal"><span class="pre">gcc</span></tt> for the main program. I wasn’t able to make <tt class="docutils literal"><span class="pre">gcc</span></tt> happy without <tt class="docutils literal"><span class="pre">LD_LIBRARY_PATH</span></tt> on Linux if the rpath starts with <tt class="docutils literal"><span class="pre">$ORIGIN</span></tt>, though.</p> </li> <li><p class="first">The same rpath trick might be used to let the main program find <tt class="docutils literal"><span class="pre">libmy_plugin.so</span></tt> in the first place without <tt class="docutils literal"><span class="pre">LD_LIBRARY_PATH</span></tt>. (This doesn’t apply if the main program uses <tt class="docutils literal"><span class="pre">dlopen()</span></tt> to load it as a dynamic plugin.) You’d make the main program with <tt class="docutils literal"><span class="pre">gcc</span> <span class="pre">-Wl,-rpath=/path/to/libmyplugin</span></tt>, possibly with <tt class="docutils literal"><span class="pre">$ORIGIN</span></tt>. The <tt class="docutils literal"><span class="pre">$</span></tt> in <tt class="docutils literal"><span class="pre">$ORIGIN</span></tt> causes various shell problems on its own: if using a common shell you need to say <tt class="docutils literal"><span class="pre">gcc</span> <span class="pre">-Wl,-rpath=\$ORIGIN</span></tt>. From a Makefile, you need to say something like <tt class="docutils literal"><span class="pre">gcc</span> <span class="pre">-Wl,-rpath=\$$ORIGIN</span></tt>.</p> </li> </ul> </div> <div class="section" id="using-multiple-cffi-made-dlls"> <h2><a class="toc-backref" href="#id12">Using multiple CFFI-made DLLs</a><a class="headerlink" href="#using-multiple-cffi-made-dlls" title="Permalink to this headline">¶</a></h2> <p>Multiple CFFI-made DLLs can be used by the same process.</p> <p>Note that all CFFI-made DLLs in a process share a single Python interpreter. The effect is the same as the one you get by trying to build a large Python application by assembling a lot of unrelated packages. Some of these might be libraries that monkey-patch some functions from the standard library, for example, which might be unexpected from other parts.</p> </div> <div class="section" id="multithreading"> <h2><a class="toc-backref" href="#id13">Multithreading</a><a class="headerlink" href="#multithreading" title="Permalink to this headline">¶</a></h2> <p>Multithreading should work transparently, based on Python’s standard Global Interpreter Lock.</p> <p>If two threads both try to call a C function when Python is not yet initialized, then locking occurs. One thread proceeds with initialization and blocks the other thread. The other thread will be allowed to continue only when the execution of the initialization-time Python code is done.</p> <p>If the two threads call two <em>different</em> CFFI-made DLLs, the Python initialization itself will still be serialized, but the two pieces of initialization-time Python code will not. The idea is that there is a priori no reason for one DLL to wait for initialization of the other DLL to be complete.</p> <p>After initialization, Python’s standard Global Interpreter Lock kicks in. The end result is that when one CPU progresses on executing Python code, no other CPU can progress on executing more Python code from another thread of the same process. At regular intervals, the lock switches to a different thread, so that no single thread should appear to block indefinitely.</p> </div> <div class="section" id="testing"> <h2><a class="toc-backref" href="#id14">Testing</a><a class="headerlink" href="#testing" title="Permalink to this headline">¶</a></h2> <p>For testing purposes, a CFFI-made DLL can be imported in a running Python interpreter instead of being loaded like a C shared library.</p> <p>You might have some issues with the file name: for example, on Windows, Python expects the file to be called <tt class="docutils literal"><span class="pre">c_module_name.pyd</span></tt>, but the CFFI-made DLL is called <tt class="docutils literal"><span class="pre">target.dll</span></tt> instead. The base name <tt class="docutils literal"><span class="pre">target</span></tt> is the one specified in <tt class="docutils literal"><span class="pre">ffibuilder.compile()</span></tt>, and on Windows the extension is <tt class="docutils literal"><span class="pre">.dll</span></tt> instead of <tt class="docutils literal"><span class="pre">.pyd</span></tt>. You have to rename or copy the file, or on POSIX use a symlink.</p> <p>The module then works like a regular CFFI extension module. It is imported with “<tt class="docutils literal"><span class="pre">from</span> <span class="pre">c_module_name</span> <span class="pre">import</span> <span class="pre">ffi,</span> <span class="pre">lib</span></tt>” and exposes on the <tt class="docutils literal"><span class="pre">lib</span></tt> object all C functions. You can test it by calling these C functions. The initialization-time Python code frozen inside the DLL is executed the first time such a call is done.</p> </div> <div class="section" id="embedding-and-extending"> <h2><a class="toc-backref" href="#id15">Embedding and Extending</a><a class="headerlink" href="#embedding-and-extending" title="Permalink to this headline">¶</a></h2> <p>The embedding mode is not incompatible with the non-embedding mode of CFFI.</p> <p>You can use <em>both</em> <tt class="docutils literal"><span class="pre">ffibuilder.embedding_api()</span></tt> and <tt class="docutils literal"><span class="pre">ffibuilder.cdef()</span></tt> in the same build script. You put in the former the declarations you want to be exported by the DLL; you put in the latter only the C functions and types that you want to share between C and Python, but not export from the DLL.</p> <p>As an example of that, consider the case where you would like to have a DLL-exported C function written in C directly, maybe to handle some cases before calling Python functions. To do that, you must <em>not</em> put the function’s signature in <tt class="docutils literal"><span class="pre">ffibuilder.embedding_api()</span></tt>. (Note that this requires more hacks if you use <tt class="docutils literal"><span class="pre">ffibuilder.embedding_api(f.read())</span></tt>.) You must only write the custom function definition in <tt class="docutils literal"><span class="pre">ffibuilder.set_source()</span></tt>, and prefix it with the macro CFFI_DLLEXPORT:</p> <div class="highlight-c"><div class="highlight"><pre><span class="n">CFFI_DLLEXPORT</span> <span class="kt">int</span> <span class="nf">myfunc</span><span class="p">(</span><span class="kt">int</span> <span class="n">a</span><span class="p">,</span> <span class="kt">int</span> <span class="n">b</span><span class="p">)</span> <span class="p">{</span> <span class="cm">/* implementation here */</span> <span class="p">}</span> </pre></div> </div> <p>This function can, if it wants, invoke Python functions using the general mechanism of “callbacks”—called this way because it is a call from C to Python, although in this case it is not calling anything back:</p> <div class="highlight-python"><div class="highlight"><pre><span class="n">ffibuilder</span><span class="o">.</span><span class="n">cdef</span><span class="p">(</span><span class="s">"""</span> <span class="s"> extern "Python" int mycb(int);</span> <span class="s">"""</span><span class="p">)</span> <span class="n">ffibuilder</span><span class="o">.</span><span class="n">set_source</span><span class="p">(</span><span class="s">"my_plugin"</span><span class="p">,</span> <span class="s">r"""</span> <span class="s"> static int mycb(int); /* the callback: forward declaration, to make</span> <span class="s"> it accessible from the C code that follows */</span> <span class="s"> CFFI_DLLEXPORT int myfunc(int a, int b)</span> <span class="s"> {</span> <span class="s"> int product = a * b; /* some custom C code */</span> <span class="s"> return mycb(product);</span> <span class="s"> }</span> <span class="s">"""</span><span class="p">)</span> </pre></div> </div> <p>and then the Python initialization code needs to contain the lines:</p> <div class="highlight-python"><div class="highlight"><pre><span class="nd">@ffi.def_extern</span><span class="p">()</span> <span class="k">def</span> <span class="nf">mycb</span><span class="p">(</span><span class="n">x</span><span class="p">):</span> <span class="k">print</span> <span class="s">"hi, I'm called with x ="</span><span class="p">,</span> <span class="n">x</span> <span class="k">return</span> <span class="n">x</span> <span class="o">*</span> <span class="mi">10</span> </pre></div> </div> <p>This <tt class="docutils literal"><span class="pre">@ffi.def_extern</span></tt> is attaching a Python function to the C callback <tt class="docutils literal"><span class="pre">mycb()</span></tt>, which in this case is not exported from the DLL. Nevertheless, the automatic initialization of Python occurs when <tt class="docutils literal"><span class="pre">mycb()</span></tt> is called, if it happens to be the first function called from C. More precisely, it does not happen when <tt class="docutils literal"><span class="pre">myfunc()</span></tt> is called: this is just a C function, with no extra code magically inserted around it. It only happens when <tt class="docutils literal"><span class="pre">myfunc()</span></tt> calls <tt class="docutils literal"><span class="pre">mycb()</span></tt>.</p> <p>As the above explanation hints, this is how <tt class="docutils literal"><span class="pre">ffibuilder.embedding_api()</span></tt> actually implements function calls that directly invoke Python code; here, we have merely decomposed it explicitly, in order to add some custom C code in the middle.</p> <p>In case you need to force, from C code, Python to be initialized before the first <tt class="docutils literal"><span class="pre">@ffi.def_extern()</span></tt> is called, you can do so by calling the C function <tt class="docutils literal"><span class="pre">cffi_start_python()</span></tt> with no argument. It returns an integer, 0 or -1, to tell if the initialization succeeded or not. Currently there is no way to prevent a failing initialization from also dumping a traceback and more information to stderr.</p> </div> </div> </div> </div> </div> <div class="sphinxsidebar"> <div class="sphinxsidebarwrapper"> <h3><a href="index.html">Table Of Contents</a></h3> <ul> <li><a class="reference internal" href="#">Using CFFI for embedding</a><ul> <li><a class="reference internal" href="#usage">Usage</a></li> <li><a class="reference internal" href="#more-reading">More reading</a></li> <li><a class="reference internal" href="#troubleshooting">Troubleshooting</a></li> <li><a class="reference internal" href="#issues-about-using-the-so">Issues about using the .so</a></li> <li><a class="reference internal" href="#using-multiple-cffi-made-dlls">Using multiple CFFI-made DLLs</a></li> <li><a class="reference internal" href="#multithreading">Multithreading</a></li> <li><a class="reference internal" href="#testing">Testing</a></li> <li><a class="reference internal" href="#embedding-and-extending">Embedding and Extending</a></li> </ul> </li> </ul> <h4>Previous topic</h4> <p class="topless"><a href="cdef.html" title="previous chapter">Preparing and Distributing modules</a></p> <h3>This Page</h3> <ul class="this-page-menu"> <li><a href="_sources/embedding.txt" rel="nofollow">Show Source</a></li> </ul> <div id="searchbox" style="display: none"> <h3>Quick search</h3> <form class="search" action="search.html" method="get"> <input type="text" name="q" /> <input type="submit" value="Go" /> <input type="hidden" name="check_keywords" value="yes" /> <input type="hidden" name="area" value="default" /> </form> <p class="searchtip" style="font-size: 90%"> Enter search terms or a module, class or function name. </p> </div> <script type="text/javascript">$('#searchbox').show(0);</script> </div> </div> <div class="clearer"></div> </div> <div class="related"> <h3>Navigation</h3> <ul> <li class="right" style="margin-right: 10px"> <a href="genindex.html" title="General Index" >index</a></li> <li class="right" > <a href="cdef.html" title="Preparing and Distributing modules" >previous</a> |</li> <li><a href="index.html">CFFI 1.9.1 documentation</a> »</li> </ul> </div> <div class="footer"> © Copyright 2012-2015, Armin Rigo, Maciej Fijalkowski. Created using <a href="http://sphinx.pocoo.org/">Sphinx</a> 1.1.3. </div> </body> </html>