{"id":2375,"date":"2018-06-14T06:00:05","date_gmt":"2018-06-14T14:00:05","guid":{"rendered":"http:\/\/blogs.msdn.microsoft.com\/pythonengineering\/?p=2375"},"modified":"2019-02-28T14:31:15","modified_gmt":"2019-02-28T22:31:15","slug":"embedding-python-in-a-cpp-project-with-visual-studio","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/python\/embedding-python-in-a-cpp-project-with-visual-studio\/","title":{"rendered":"Embedding Python in a C++ project with Visual Studio"},"content":{"rendered":"<p>Watch the video version of this post on VS Toolbox\n<iframe width=\"560\" height=\"315\" src=\"https:\/\/channel9.msdn.com\/Shows\/Visual-Studio-Toolbox\/Embedding-Python-in-a-C-Project\/player\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\"><\/iframe><\/p>\n<h2>Let&#8217;s get started<\/h2>\n<p><a href=\"http:\/\/devblogs.microsoft.com\/python\/wp-content\/uploads\/sites\/12\/2018\/06\/Sinbad.png\"><img decoding=\"async\" class=\"alignright size-medium wp-image-2385\" src=\"https:\/\/devblogs.microsoft.com\/python\/wp-content\/uploads\/sites\/12\/2018\/06\/Sinbad-300x240-1.png\" alt=\"Sinbad the ogre dancing in the Ogre3d game engine\" width=\"300\" height=\"240\" \/><\/a><\/p>\n<p>In this post, we&#8217;re going to walk through a sample project that demonstrates scripting a C++ application with Python using CPython, PyBind11 and Visual Studio 2017. The sample code is available at <a href=\"https:\/\/github.com\/zooba\/ogre3d-python-embed\">github.com\/zooba\/ogre3d-python-embed<\/a> and setup instructions are below.<\/p>\n<p>Ogre3d is an open-source game engine written in C++ that has been used in games such as <a href=\"https:\/\/www.youtube.com\/watch?v=EljXnVDHwwI\">Hob<\/a> and <a href=\"https:\/\/www.youtube.com\/watch?v=HH_sGQRSpzo\">Torchlight 2<\/a>. Both the engine and its source code are freely available from <a href=\"https:\/\/www.ogre3d.org\/\">their website<\/a>. For this sample, we have taken one of their character animation demos and extended it with Python. Rather than using the keyboard to move the character around, we can use Python code to call into the C++ functions that control him.<\/p>\n<p>To build and run this sample on your own machine, you will require <strong>Visual Studio 2017<\/strong> with the <strong>Python workload<\/strong>, the <strong>Python Native Development<\/strong> option, and <strong>Python 3.6 32-bit<\/strong>. If you already have Visual Studio 2017, these can be added by launching &#8220;Visual Studio Installer&#8221; and modifying the existing install.<\/p>\n<blockquote><p><small>Note: When you install Python 3.6 32-bit through Visual Studio, it automatically includes debugging symbols. If you install it yourself, you will need to select &#8220;Customize installation&#8221; and include debugging symbols. If you have already installed it, you can use Programs and Features to modify your install and add debugging symbols.<\/small><\/p><\/blockquote>\n<p>Clone our repository using <tt>git clone --recurse-submodules https:\/\/github.com\/zooba\/ogre3d-python-embed.git<\/tt> or using Team Explorer in Visual Studio. There is a PowerShell script in the root of the repository called <tt>get_externals.ps1<\/tt> that will download and extract the version of the Ogre3d and SDL2 runtimes needed, and will prompt if you are missing either Python 3.6 32-bit or the DirectX redistributables (you can download the latter <a href=\"https:\/\/www.microsoft.com\/en-us\/download\/details.aspx?id=35\">here<\/a>, but be aware that this is a very old installer that will offer to install a browser toolbar &#8211; feel free to deselect that).<\/p>\n<p>Once everything is installed, open <tt>srcPythonCharacter.sln<\/tt> in Visual Studio 2017 and press Ctrl+F5 to build and run the sample. While running, the sample will capture your mouse cursor, but you can use Alt+Tab to switch to another window. We will do that next to look at some of the code.<\/p>\n<p>In Visual Studio, open Solution Explorer and then open the following files. We will be looking at each in the next few sections.<\/p>\n<ul>\n<li><tt>ogre.pyi<\/tt><\/li>\n<li><tt>ogre_module.h<\/tt><\/li>\n<li><tt>SinbadCharacterController.h<\/tt><\/li>\n<li><tt>ai.py<\/tt><\/li>\n<\/ul>\n<h2>Modifying without recompiling<\/h2>\n<p><a href=\"http:\/\/devblogs.microsoft.com\/python\/wp-content\/uploads\/sites\/12\/2018\/06\/Sinbad_Script.png\"><img decoding=\"async\" class=\"alignright size-medium wp-image-2435\" src=\"https:\/\/devblogs.microsoft.com\/python\/wp-content\/uploads\/sites\/12\/2018\/06\/Sinbad_Script-300x211-1.png\" alt=\"Simple Python list containing dance, stand, and more steps\" width=\"300\" height=\"211\" \/><\/a><\/p>\n<p>The Python module <tt>ai.py<\/tt> is where we define the behavior of Sinbad, our dancing ogre. Collapse the command definitions region by clicking the &#8220;-&#8221; symbol to the left of <tt>#region Command definitions<\/tt> and look at the <tt>SCRIPT<\/tt> variable. Each item in this list is the movement we want Sinbad to do, including the amount of time he should do it for. There are already some extra movements listed but commented out, so try uncommenting them or adding your own. You can do this while the demo is running in the background.<\/p>\n<p>Once you&#8217;ve made changes, save this file, use Alt+Tab to go back to the running sample, and press F5. The F5 key will reload the script, and you will see your changes (or an error) immediately. Compare this to normal C++ development, where you would have had to stop running, modify the code, recompile (and wait!), start running again, and return to the same state you were previously.<\/p>\n<p>This is possible because of CPython&#8217;s support for <a href=\"https:\/\/docs.python.org\/3\/extending\/embedding.html\">embedding<\/a>, and made simple by the powerful <a href=\"https:\/\/github.com\/pybind\/pybind11\">pybind11<\/a> library. Embedding allows you to host the Python runtime in any native application, on any platform and using any compiler supported by CPython. So rather than launching &#8220;python.exe&#8221; with a script, you can load python36.dll into your own application and use it directly.<\/p>\n<p>It is very easy to make Python representations of your C++ classes with pybind11. Switch to the <tt>ogre_module.h<\/tt> file to see what we did for this sample.<\/p>\n<p><a href=\"http:\/\/devblogs.microsoft.com\/python\/wp-content\/uploads\/sites\/12\/2018\/06\/Sinbad_Module.png\"><img decoding=\"async\" class=\"alignnone size-large wp-image-2425\" src=\"https:\/\/devblogs.microsoft.com\/python\/wp-content\/uploads\/sites\/12\/2018\/06\/Sinbad_Module-1024x524-1.png\" alt=\"Exposing a C++ class to Python using the pybind11 library\" width=\"1024\" height=\"524\" \/><\/a><\/p>\n<p>This file defines the mapping between a Python module named &#8220;ogre&#8221; and the classes and functions we want to make available. Using the metaprogramming features added in C++11, the pybind11 library automatically generates code to do the type conversions necessary to make Python code transparently interact with your C++ code. If you switch back to <tt>ai.py<\/tt>, you will see that the <tt>ogre.CharacterController<\/tt> class is imported and used in Python code to call back into C.<\/p>\n<p>But how can we be sure that it is really doing all this? It seems pretty magical, and barely enough work on our part to make a game engine suddenly support Python scripting. In the next section, we will look at the proof that it is doing what we claim.<\/p>\n<h2>Debugging Python and C++<\/h2>\n<p>If you&#8217;ve got the demo running, now is the time to exit it by clicking on the Stop Debugging button. Visual Studio is famous for its debugging features, and this one is pretty cool. When you installed the Python Native Development option, we included the ability to do mixed Python\/C++ debugging, whether you&#8217;re in a Python project or a C++ project. You can find information about doing this from Python project in <a href=\"https:\/\/docs.microsoft.com\/visualstudio\/python\/debugging-mixed-mode-c-cpp-python-in-visual-studio\">our documentation<\/a>, but in this case we are going to launch a C++ project with Python debugging enabled.<\/p>\n<p><a href=\"http:\/\/devblogs.microsoft.com\/python\/wp-content\/uploads\/sites\/12\/2018\/06\/Sinbad_Debugging.png\"><img decoding=\"async\" class=\"alignleft size-medium wp-image-2395\" src=\"https:\/\/devblogs.microsoft.com\/python\/wp-content\/uploads\/sites\/12\/2018\/06\/Sinbad_Debugging-300x144-1.png\" alt=\"Selecting Python\/Native Debugging from the dropdown in Visual Studio 2017\" width=\"300\" height=\"144\" \/><\/a><\/p>\n<p>Find the Start debugging button on the toolbar. Depending on your settings and the file you have open, it be labelled &#8220;Start&#8221;, &#8220;Local Windows Debugger&#8221;, &#8220;Python\/Native Debugging&#8221; or &#8220;Attach&#8230;&#8221;. Clicking the small down arrow next to the button will display your options.<\/p>\n<p>Select &#8220;Python\/Native Debugging&#8221; here to make it your new default. All the settings required to make the sample correctly load are already configured for this project (see <a href=\"https:\/\/github.com\/zooba\/ogre3d-python-embed\/blob\/master\/EmbeddingPython.md\">this page<\/a> for the details), so you can now press F5 to launch the sample again, but this time with the debugger attached.<\/p>\n<p><a href=\"http:\/\/devblogs.microsoft.com\/python\/wp-content\/uploads\/sites\/12\/2018\/06\/Sinbad_DebugWindows.png\"><img decoding=\"async\" class=\"alignright size-medium wp-image-2405\" src=\"https:\/\/devblogs.microsoft.com\/python\/wp-content\/uploads\/sites\/12\/2018\/06\/Sinbad_DebugWindows-300x144-1.png\" alt=\"Debugging Python and C++ simultaneously in Visual Studio 2017\" width=\"300\" height=\"144\" \/><\/a><\/p>\n<p>Open <tt>ai.py<\/tt> again and set a breakpoint in the <tt>on_frame<\/tt> function near the end of the file. This function is called for each frame, but normally returns quickly until it&#8217;s time to run the next step in the script. So while the demo is running, sooner or later this function will be hit. When it is, you&#8217;ll see a mixed call stack showing both Python and C++ frames. (In this screenshot, I&#8217;ve hidden external code, so you may see some extra frames from pybind11.)<\/p>\n<p>As you press F11 to step through the code, you will jump between Python and C++ just as naturally as either language on its own, and anywhere you view a Python object you&#8217;ll be able to see the regular Python view. Breakpoints set in native code or Python code will be hit, and switching up and down the call stack will let you view both kinds of frames.<\/p>\n<h2>Type hints for Python code<\/h2>\n<p>Finally, let&#8217;s take another look at the Python code and how Visual Studio helps you be a productive developer. In general, when you have defined a Python class in C++ you are not going to get IntelliSense when editing the code using it. This is unfortunate, as IntelliSense is most useful when using code that does not have easily read sources, but we now have the ability to provide IntelliSense separately.<\/p>\n<p><a href=\"http:\/\/devblogs.microsoft.com\/python\/wp-content\/uploads\/sites\/12\/2018\/06\/Sinbad_TypeStub.png\"><img decoding=\"async\" class=\"alignright size-medium wp-image-2445\" src=\"https:\/\/devblogs.microsoft.com\/python\/wp-content\/uploads\/sites\/12\/2018\/06\/Sinbad_TypeStub-300x106-1.png\" alt=\"Snippet from the type stub file defining ogre.CharacterController\" width=\"300\" height=\"106\" \/><\/a><\/p>\n<p>Open the <tt>ogre.pyi<\/tt> file. While it looks very much like Python code, it&#8217;s actually a type stub file that exists solely for its <a href=\"https:\/\/docs.microsoft.com\/visualstudio\/python\/editing-python-code-in-visual-studio#type-hints\">type hints<\/a>. We include class and function definitions, but no bodies. With function annotations, we can specify the expected types and the return type of each function, and we will also extract and present documentation strings.<\/p>\n<p>As this file is named <tt>ogre.pyi<\/tt>, when the <tt>ai.py<\/tt> module imports the native <tt>ogre<\/tt> module, IntelliSense will import the type stub instead. Without the type stub, we would not be able to resolve the ogre module at all, as it is generated at runtime by pybind11 and there is no other way for us to find it.<\/p>\n<p>Switch back to <tt>ai.py<\/tt> and find the <tt>on_frame<\/tt> function. As this is called from C++, we have no information about the arguments that are passed to it, so to get IntelliSense we use type annotations to specify the parameter types. If you start typing <tt>character.<\/tt> within this function then you will see all the members that were specified in the type stub.<\/p>\n<p><a href=\"http:\/\/devblogs.microsoft.com\/python\/wp-content\/uploads\/sites\/12\/2018\/06\/Sinbad_IntelliSense.png\"><img decoding=\"async\" class=\"alignnone size-large wp-image-2415\" src=\"https:\/\/devblogs.microsoft.com\/python\/wp-content\/uploads\/sites\/12\/2018\/06\/Sinbad_IntelliSense-1024x347-1.png\" alt=\"Python IntelliSense through a type stub file\" width=\"1024\" height=\"347\" \/><\/a><\/p>\n<p>While this is very convenient and necessary in some situations, most of the time we are able to show you good IntelliSense without needing type hints. If you hover over <tt>command<\/tt> in this function you will see all the possible command types; using Go To Definition (F12) on <tt>command.execute<\/tt> will take you to either of the implementations of that function; and even though there are no type hints on these functions, we are still able to provide all the completions on <tt>character<\/tt> through our type analysis engine. Type stub files are useful for when you want to provide users with IntelliSense for code that cannot be automatically analyzed, and annotations can fill in the gaps that occur when using complex or unusual Python code.<\/p>\n<h2>Summary<\/h2>\n<p>If you develop native applications that have frequently-changing logic or business rules, it is easy to move that logic from C or C++ into Python, where it can be easily modified and updated without needing to recompile or restart your application. The combination of Visual Studio 2017 with the official CPython releases and pybind11 is the most productive way to develop and debug all aspects of your hybrid C++ and Python project. Download the free <a href=\"https:\/\/visualstudio.com\/vs\/features\/python\">Visual Studio 2017 Community<\/a> to get started today.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In this post, we&#8217;re going to walk through a sample project that demonstrates scripting a C++ application with Python using CPython, PyBind11 and Visual Studio 2017. We show how you can wrap a C++ class with Python and how to use cross-language debugging and type-hints to get a development experience that only Visual Studio can offer.<\/p>\n","protected":false},"author":1333,"featured_media":2395,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-2375","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-python"],"acf":[],"blog_post_summary":"<p>In this post, we&#8217;re going to walk through a sample project that demonstrates scripting a C++ application with Python using CPython, PyBind11 and Visual Studio 2017. We show how you can wrap a C++ class with Python and how to use cross-language debugging and type-hints to get a development experience that only Visual Studio can offer.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/python\/wp-json\/wp\/v2\/posts\/2375","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/python\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/python\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/python\/wp-json\/wp\/v2\/users\/1333"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/python\/wp-json\/wp\/v2\/comments?post=2375"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/python\/wp-json\/wp\/v2\/posts\/2375\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/python\/wp-json\/wp\/v2\/media\/2395"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/python\/wp-json\/wp\/v2\/media?parent=2375"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/python\/wp-json\/wp\/v2\/categories?post=2375"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/python\/wp-json\/wp\/v2\/tags?post=2375"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}