{"id":1995,"date":"2017-12-19T12:36:30","date_gmt":"2017-12-19T20:36:30","guid":{"rendered":"http:\/\/blogs.msdn.microsoft.com\/commandline\/?p=1995"},"modified":"2019-02-18T13:28:45","modified_gmt":"2019-02-18T21:28:45","slug":"af_unix-comes-to-windows","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/commandline\/af_unix-comes-to-windows\/","title":{"rendered":"AF_UNIX comes to Windows"},"content":{"rendered":"<h2><span>Introduction:<\/span><span>\u00a0<\/span><\/h2>\n<p><span>Beginning in <a href=\"https:\/\/blogs.windows.com\/windowsexperience\/2017\/12\/19\/announcing-windows-10-insider-preview-build-17063-pc\/\">Insider Build <\/a><\/span><span>17063<\/span><span>, you\u2019ll be able to use the <\/span><span>unix<\/span><span> socket (<\/span><span>AF_UNIX<\/span><span>)<\/span><span> address family on Windows<\/span><span> to communicate between <\/span><span>W<\/span><span>in32 processes<\/span><span>. <\/span><a href=\"http:\/\/man7.org\/linux\/man-pages\/man7\/unix.7.html\"><span>Unix<\/span><\/a> <span>sockets allow inter-process communication<\/span><span> (IPC)<\/span><span> between process<\/span><span>es<\/span><span> on the same machine.<\/span><span>\u00a0<\/span><\/p>\n<h2><span>Overview<\/span><span>:<\/span><span>\u00a0<\/span><\/h2>\n<p><span>Support for the <\/span><span>unix<\/span><span> socket<\/span><span> has existed both in BSD and Linux for the longest time, but, not on Windows.<\/span><span> On Windows, there were some alternatives for local IPC, such as <\/span><a href=\"https:\/\/msdn.microsoft.com\/en-us\/library\/windows\/desktop\/aa365590(v=vs.85).aspx\"><span>named pipes<\/span><\/a><span>.<\/span><span> But, calling conventions are different between the named pipes and sockets<\/span><span>, making writing low-maintenance cross-platform applications difficult. <\/span><span>For<\/span><span> example, <\/span><span>one such place <\/span><span>where these two constructs differ (other than the API) is terminating the connection. BSD Socket API provides a bidirectional close semantics using <code>&lt;\/span&gt;&lt;a href=\"http:\/\/man7.org\/linux\/man-pages\/man2\/shutdown.2.html\"&gt;&lt;span&gt;shutdown&lt;\/span&gt;&lt;\/a&gt;&lt;span&gt;<\/code>. There is no direct equivalent of that in named pipes.<\/span> <span>Such differences <\/span><span>make it difficult to<\/span> <span>port<\/span> <span>unix<\/span><span> socket <\/span><span>applications<\/span><span> from Linux to Windows <\/span><span>and vice versa;<\/span> <span>up until now!<\/span><span>\u00a0<\/span><\/p>\n<p><span>Build <\/span><span>17063<\/span> <span>brings native support for the <\/span><span>unix<\/span><span> socket <\/span><span>to Windows. Starting this build, two Win<\/span><span>32<\/span><span> process<\/span><span>es<\/span><span> can use the AF_UNIX address family over <\/span><a href=\"https:\/\/msdn.microsoft.com\/en-us\/library\/windows\/desktop\/ms738545(v=vs.85).aspx\"><span>Winsock API<\/span><\/a><span> (which is very similar to the BSD socket API) to communicate with each other.<\/span><span> Currently, the support only exists for the <\/span><span>stream (<\/span><span>SOCK_STREAM<\/span><span>)<\/span><span> socket type<\/span><span>, which is a connection-oriented protocol for one-to-one communication. Support for the datagram (SOCK_DGRAM) can be considered in future depending on the <\/span><span>adoption, feedback and scenario<\/span><span>s<\/span><span>.<\/span><span>\u00a0<\/span><\/p>\n<h2><span>Addressing scheme &#8211; <\/span><span>sockaddr_un<\/span><span>:<\/span><span>\u00a0<\/span><\/h2>\n<p><span>The \u2018<\/span><span>sockaddr_un<\/span><span>\u2019 structure is used for defining the <\/span><span>address of a <\/span><span>unix<\/span><span> socket.<\/span><span> In the Windows implementation of the <\/span><span>unix<\/span><span> socket, we have kept the name, definition and semantics of the <\/span><span>unix<\/span><span> socket address the same as that in Linux<\/span><span>, to make cross-platform development easier<\/span><span>.<\/span><span>\u00a0<\/span><\/p>\n<p><span>There are three different addressing <\/span><span>formats<\/span><span> for <\/span><span>unix<\/span><span> sockets. <\/span><span>\u2018pathname\u2019, \u2018abstract\u2019 and \u2018unnamed\u2019 sockets. <\/span><span>\u2018pathname\u2019 sockets are bound to the filesystem<\/span><span>, where<\/span> <span>in <\/span><span>the \u2018<\/span><span>sun_path<\/span><span>\u2019 member of the struct is used to specify a null-terminated UTF-8 file system path. You can expect the same from the Windows implementation<\/span><span>,<\/span><span> where \u2018<\/span><span>sun_path<\/span><span>\u2019 specifies a Win32 UTF-8 file system path.<\/span><span>\u00a0<\/span><\/p>\n<p><span>The second category is the \u2018abstract\u2019 socket address where the <\/span><span>first character in \u2018<\/span><span>sun_path<\/span><span>\u2019 is a <\/span><i><span>null<\/span><\/i><span> byte.<\/span><span> Windows implementation of AF_UNIX socket can also accept abstract addresses. The one difference noteworthy here is that the Windows <\/span><span>unix<\/span><span> socket <\/span><span>implementation <\/span><span>currently does not support <\/span><span>the <\/span><span>auto<\/span><span>bind<\/span> <span>feature whereby an abstract address is auto-generated by the implementation on behalf of the user.<\/span><span>\u00a0<\/span><\/p>\n<p><span>Lastly, \u2018unnamed\u2019 sockets, where the socket is bound to a pathname with no name.<\/span><span> This is also supported on Windows <\/span><span>unix<\/span><span> socket implementation<\/span><span>. Although, one of the socket API\u2019s <\/span><a href=\"http:\/\/man7.org\/linux\/man-pages\/man2\/socketpair.2.html\"><span>socketpair<\/span><\/a> <span>that generates unnamed sockets, itself is not supported in Winsock<\/span><span> 2.0<\/span><span>.<\/span><span>\u00a0<\/span><\/p>\n<h2><span>Security:<\/span><span>\u00a0<\/span><\/h2>\n<p><span>Unix sockets provide a mechanism for secure communication.<\/span><span> C<\/span><span>ommunication <\/span><span>over <\/span><span>unix<\/span><span> sockets<\/span> <span>can be secured by controlling the file<\/span><span> (or directory)<\/span><span> permissions on the pathname sockets<\/span> <span>(<\/span><span>or the parent directory<\/span><span>)<\/span><span>. <\/span><span>For example, <\/span><span>the <\/span><i><span>bind<\/span><\/i><span> socket API creates a \u2018socket\u2019 file with the given pathname<\/span><span>. <\/span><span>The creation of the new socket file will fail if the calling process does not has write permission on the directory where the file is being created. <\/span><span>Similarly, for connecting to a stream socket, the connecting process should have write permission on the socket. <\/span><span>The same level of security is available and enforced on <\/span><span>the <\/span><span>Windows <\/span><span>unix<\/span><span> socket<\/span><span> implementation<\/span><span>. <\/span><span>See the man page on AF_UNIX for more details<\/span><span> on the security<\/span><span>.<\/span><span>\u00a0<\/span><\/p>\n<h2><span>Implementation:<\/span><span>\u00a0<\/span><\/h2>\n<p><span>Majority of the functionality for the Windows AF_UNIX is implemented in the Windows kernel in a driver<\/span><span>:<\/span> <i><span>afunix.sys<\/span><\/i><span>. <\/span><span>The <\/span><span>Windows kernel <\/span><span>networking <\/span><span>stack provides a very pluggable and extensible model, that makes it easy to support new network providers.<\/span><span> The socket file itself <\/span><span>that is created as part of the bind call <\/span><span>is a custom <\/span><a href=\"https:\/\/en.wikipedia.org\/wiki\/NTFS_reparse_point\"><span>NTFS reparse point<\/span><\/a><span>.<\/span><span>\u00a0<\/span><\/p>\n<h2><span>Unsupported<\/span><span>\\unavailable<\/span><span>:<\/span><span>\u00a0<\/span><\/h2>\n<p><span>Summarizing from the above, the following Linux <\/span><span>unix<\/span><span> socket<\/span><span> features are either <\/span><b><span>currently<\/span><\/b><span> unavailable or unsupported in the Windows <\/span><span>unix<\/span><span> socket<\/span><span> implementation.<\/span><span>\u00a0<\/span><\/p>\n<ul>\n<li><span>AF_UNIX datagram (SOCK_DGRAM) or sequence packet (SOCK_SEQPACKET) <\/span><span>socket <\/span><span>type.<\/span><span>\u00a0<\/span><\/li>\n<li><span>Ancillary data: Linux<\/span><span>&#8216;s<\/span> <span>unix<\/span><span> socket <\/span><span>implementation<\/span> <span>supports passing ancillary data such as <\/span><span>passing file descriptors (<\/span><span>`SCM_RIGHTS`<\/span><span>)<\/span> <span>or credentials (\u2018<\/span><span>SCM_CREDENTIALS`<\/span><span>)<\/span><span> over <\/span><span>the <\/span><span>socket<\/span><span>. There is no support for ancillary data in <\/span><span>the <\/span><span>Windows <\/span><span>unix<\/span><span> socket <\/span><span>implementation<\/span><span>.<\/span><span>\u00a0<\/span><\/li>\n<\/ul>\n<ul>\n<li><span>Autobind<\/span><span> feature<\/span><span> (see the section on \u2018<\/span><span>sockaddr_un<\/span><span>\u2019 for details).<\/span><span>\u00a0<\/span><\/li>\n<li><span>socketpair<\/span><span>: <\/span><span>socketpair<\/span><span> socket API is not supported in Winsock<\/span><span> 2.0.<\/span><span>\u00a0<\/span><\/li>\n<\/ul>\n<h2><span>How can I <\/span><span>write a<\/span><span> Windows AF_UNIX<\/span><span> app<\/span><span>?<\/span><span>\u00a0<\/span><\/h2>\n<ol>\n<li><span>Download the <\/span><span>Windows Insiders <\/span><span>SDK f<\/span><span>or<\/span><span> the Windows build <\/span><span>17061 &#8212; <a href=\"https:\/\/www.microsoft.com\/en-us\/software-download\/windowsinsiderpreviewSDK\">available here<\/a><\/span><span>.<\/span><span>\u00a0<\/span><span>\u00a0<\/span><\/li>\n<li><span>Check whether your Windows build has support for <\/span><span>unix<\/span><span> socket<\/span><span> by running \u201c<\/span><i><span>sc<\/span><\/i><i><span> query <\/span><\/i><i><span>afunix<\/span><\/i><span>\u201d from a Windows admin command prompt.<\/span><span>\u00a0<\/span><\/li>\n<li><span>#include &lt;<\/span><span>afunix.h<\/span><span>&gt;<\/span><span> in your Windows application<\/span><span> and w<\/span><span>rite a Windows <\/span><span>unix<\/span><span> socket<\/span> <span>winsock<\/span><span> application as you would write any other <\/span><span>unix<\/span><span> socket<\/span><span> application, but, using <\/span><a href=\"https:\/\/msdn.microsoft.com\/en-us\/library\/windows\/desktop\/ms738545(v=vs.85).aspx\"><span>Winsock API<\/span><span>\u2019s<\/span><\/a><span>.<\/span><span>\u00a0<\/span><\/li>\n<\/ol>\n<p><span>Note: <\/span><span>As mentioned above<\/span><span> in the \u2018<\/span><span>security\u2019 section<\/span><span>, when a socket binds a socket to a valid pathname address, a socket file is created within the filesystem.<\/span><span> On Linux, the application is expected to <\/span><a href=\"http:\/\/man7.org\/linux\/man-pages\/man2\/unlink.2.html\"><span>unlink<\/span><\/a><span> (see the notes section in the <\/span><a href=\"http:\/\/man7.org\/linux\/man-pages\/man7\/unix.7.html\"><span>man page for AF_UNIX<\/span><\/a><span>)<\/span><span> before any other socket can be bound to the same address.<\/span><span> The same applies to Windows <\/span><span>unix<\/span><span> sockets<\/span><span>, except that, <\/span><a href=\"https:\/\/msdn.microsoft.com\/en-us\/library\/windows\/desktop\/aa363915%28v=vs.85%29.aspx?f=255&amp;MSPPError=-2147217396\"><span>DeleteFile<\/span><\/a><span> (or any other file delete API) should be used to delete the socket file prior to calling <\/span><i><span>bind<\/span><\/i><span> with the same path.<\/span><span>\u00a0<\/span><\/p>\n<h2><span>What\u2019s next?<\/span><span>\u00a0<\/span><\/h2>\n<p><span>We are listening. Please try out the Windows <\/span><span>unix<\/span><span> socket provider and let us know what works, what doesn\u2019t work, what you like, what you don\u2019t like or what <\/span><span>improvements would you like<\/span><span>.\u00a0 For now, the best way to provide feedback is either via Feedback Hub under apps -&gt; Hyper-V, the WSL GitHub <a href=\"https:\/\/github.com\/Microsoft\/WSL\/issues\">issue tracker<\/a>, or as a comment on this blog.<\/span><\/p>\n<p><span>And, if you are wondering, there is already support for <\/span><span>unix<\/span><span> socket <\/span><span>within <\/span><a href=\"https:\/\/docs.microsoft.com\/en-us\/windows\/wsl\/about\"><span>Windows Subsystem for Linux<\/span><\/a><span> (WSL), how does that work with the Windows <\/span><span>un<\/span><span>ix s<\/span><span>ocket<\/span> <span>implementation? Well, currently, it doesn\u2019t, but stay tuned!<\/span><span>\u00a0<\/span><\/p>\n<p>&#8211;\u00a0Sunil Muthuswamy and the WSL Team<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Introduction:\u00a0 Beginning in Insider Build 17063, you\u2019ll be able to use the unix socket (AF_UNIX) address family on Windows to communicate between Win32 processes. Unix sockets allow inter-process communication (IPC) between processes on the same machine.\u00a0 Overview:\u00a0 Support for the unix socket has existed both in BSD and Linux for the longest time, but, not [&hellip;]<\/p>\n","protected":false},"author":1027,"featured_media":11466,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[3,5,7,9],"tags":[18,43,54,72],"class_list":["post-1995","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-linux-tools","category-windows-10","category-windows-server","category-bash-on-ubuntu-on-windows","tag-af_unix","tag-linuxtools","tag-sockets","tag-wsl"],"acf":[],"blog_post_summary":"<p>Introduction:\u00a0 Beginning in Insider Build 17063, you\u2019ll be able to use the unix socket (AF_UNIX) address family on Windows to communicate between Win32 processes. Unix sockets allow inter-process communication (IPC) between processes on the same machine.\u00a0 Overview:\u00a0 Support for the unix socket has existed both in BSD and Linux for the longest time, but, not [&hellip;]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/commandline\/wp-json\/wp\/v2\/posts\/1995","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/commandline\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/commandline\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/commandline\/wp-json\/wp\/v2\/users\/1027"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/commandline\/wp-json\/wp\/v2\/comments?post=1995"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/commandline\/wp-json\/wp\/v2\/posts\/1995\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/commandline\/wp-json\/wp\/v2\/media\/11466"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/commandline\/wp-json\/wp\/v2\/media?parent=1995"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/commandline\/wp-json\/wp\/v2\/categories?post=1995"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/commandline\/wp-json\/wp\/v2\/tags?post=1995"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}