Announcing Entity Framework Core 3.0 and Entity Framework 6.3 General Availability

Avatar

Diego

We are extremely excited to announce the general availability of EF Core 3.0and EF 6.3 on nuget.org.

The final versions of .NET Core 3.0 and ASP.NET Core 3.0 are also available now.

How to get EF Core 3.0

EF Core 3.0 is distributed exclusively as a set of NuGet packages. For example, to add the SQL Server provider to your project, you can use the following command using the dotnet tool:

dotnet add package Microsoft.EntityFrameworkCore.SqlServer --version 3.0.0

When upgrading applications that target older versions of ASP.NET Core to 3.0, you also have to add the EF Core packages as an explicit dependency.

Also starting in 3.0, the dotnet ef command-line tool is no longer included in the .NET Core SDK. Before you can execute EF Core migration or scaffolding commands, you’ll have to install this package as either a global or local tool. To install the final version of our 3.0.0 tool as a global tool, use the following command:

dotnet tool install --global dotnet-ef --version 3.0.0

Specifying the version in the command is now optional. If you omit it, dotnet tool install will automatically install the latest stable version, which is right now 3.0.0.

It’s possible to use this new version of dotnet ef with projects that use older versions of the EF Core runtime. However, older versions of the tool will not work with EF Core 3.0.

What’s new in EF Core 3.0

Including major features, minor enhancements, and bug fixes, EF Core 3.0 contains more than 600 product improvements. Here are some of the most important ones:

LINQ overhaul

We rearchitected our LINQ provider to enable translating more query patterns into SQL, generating efficient queries in more cases, and preventing inefficient queries from going undetected. The new LINQ provider is the foundation over which we’ll be able to offer new query capabilities and performance improvements in future releases, without breaking existing applications and data providers.

Restricted client evaluation

The most important design change has to do with how we handle LINQ expressions that cannot be converted to parameters or translated to SQL.

In previous versions, EF Core identified what portions of a query could be translated to SQL, and executed the rest of the query on the client. This type of client-side execution is desirable in some situations, but in many other cases it can result in inefficient queries.

For example, if EF Core 2.2 couldn’t translate a predicate in a Where() call, it executed an SQL statement without a filter, transferred all the rows from the database, and then filtered them in-memory:

var specialCustomers = 
  context.Customers
    .Where(c => c.Name.StartsWith(n) && IsSpecialCustomer(c));

That may be acceptable if the database contains a small number of rows but can result in significant performance issues or even application failure if the database contains a large number or rows.

In EF Core 3.0, we’ve restricted client evaluation to only happen on the top-level projection (essentially, the last call to Select()). When EF Core 3.0 detects expressions that can’t be translated anywhere else in the query, it throws a runtime exception.

To evaluate a predicate condition on the client as in the previous example, developers now need to explicitly switch evaluation of the query to LINQ to Objects:

var specialCustomers =
  context.Customers
    .Where(c => c.Name.StartsWith(n)) 
    .AsEnumerable() // switches to LINQ to Objects
    .Where(c => IsSpecialCustomer(c));

See the breaking changes documentation for more details about how this can affect existing applications.

Single SQL statement per LINQ query

Another aspect of the design that changed significantly in 3.0 is that we now always generate a single SQL statement per LINQ query. In previous versions, we used to generate multiple SQL statements in certain cases, like to translate Include() calls on collection navigation properties and to translate queries that followed certain patterns with subqueries. Although this was in some cases convenient, and for Include() it even helped avoid sending redundant data over the wire, the implementation was complex, it resulted in some extremely inefficient behaviors (N+1 queries), and there was situations in which the data returned across multiple queries could be inconsistent.

Similarly to client evaluation, if EF Core 3.0 can’t translate a LINQ query into a single SQL statement, it throws a runtime exception. But we made EF Core capable of translating many of the common patterns that used to generate multiple queries to a single query with JOINs.

Cosmos DB support

The Cosmos DB provider for EF Core enables developers familiar with the EF programing model to easily target Azure Cosmos DB as an application database. The goal is to make some of the advantages of Cosmos DB, like global distribution, “always on” availability, elastic scalability, and low latency, even more accessible to .NET developers. The provider enables most EF Core features, like automatic change tracking, LINQ, and value conversions, against the SQL API in Cosmos DB.

See the Cosmos DB provider documentation for more details.

C# 8.0 support

EF Core 3.0 takes advantage of a couple of the new features in C# 8.0:

Asynchronous streams

Asynchronous query results are now exposed using the new standard IAsyncEnumerable<T> interface and can be consumed using await foreach.

var orders = 
  from o in context.Orders
  where o.Status == OrderStatus.Pending
  select o;

await foreach(var o in orders.AsAsyncEnumerable())
{
  Process(o);
}

See the asynchronous streams in the C# documentation for more details.

Nullable reference types

When this new feature is enabled in your code, EF Core examines the nullability of reference type properties and applies it to corresponding columns and relationships in the database: properties of non-nullable references types are treated as if they had the [Required] data annotation attribute.

For example, in the following class, properties marked as of type string? will be configured as optional, whereas string will be configured as required:

public class Customer
{
  public int Id { get; set; }
  public string FirstName { get; set; }
  public string LastName { get; set; }
  public string? MiddleName { get; set; }
}

See nullable reference types in the C# documentation for more details.

Interception of database operations

The new interception API in EF Core 3.0 allows providing custom logic to be invoked automatically whenever low-level database operations occur as part of the normal operation of EF Core. For example, when opening connections, committing transactions, or executing commands.

Similarly to the interception features that existed in EF 6, interceptors allow you to intercept operations before or after they happen. When you intercept them before they happen, you are allowed to by-pass execution and supply alternate results from the interception logic.

For example, to manipulate command text, you can create an IDbCommandInterceptor:

public class HintCommandInterceptor : DbCommandInterceptor
{
  public override InterceptionResult ReaderExecuting(
    DbCommand command, 
    CommandEventData eventData, 
    InterceptionResult result)
  {
    // Manipulate the command text, etc. here...
    command.CommandText += " OPTION (OPTIMIZE FOR UNKNOWN)";
    return result;
  }
}

And register it with your DbContext:

services.AddDbContext(b => b
  .UseSqlServer(connectionString)
  .AddInterceptors(new HintCommandInterceptor()));

Reverse engineering of database views

Query types, which represent data that can be read from the database but not updated, have been renamed to keyless entity types. As they are an excellent fit for mapping database views in most scenarios, EF Core now automatically creates keyless entity types when reverse engineering database views.

For example, using the dotnet ef command-line tool you can type:

dotnet ef dbcontext scaffold "Server=(localdb)\mssqllocaldb;Database=Blogging;Trusted_Connection=True;" Microsoft.EntityFrameworkCore.SqlServer

And the tool will now automatically scaffold types for views and tables without keys:

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
  modelBuilder.Entity<Names>(entity =>
  {
    entity.HasNoKey();
    entity.ToView("Names");
  });

  modelBuilder.Entity<Things>(entity =>
  {
    entity.HasNoKey();
  });
}

Dependent entities sharing a table with principal are now optional

Starting with EF Core 3.0, if OrderDetails is owned by Order or explicitly mapped to the same table, it will be possible to add an Order without an OrderDetails and all of the OrderDetails properties, except the primary key will be mapped to nullable columns.

When querying, EF Core will set OrderDetails to null if any of its required properties doesn’t have a value, or if it has no required properties besides the primary key and all properties are null.

public class Order
{
    public int Id { get; set; }
    public int CustomerId { get; set; }
    public OrderDetails Details { get; set; }
}

[Owned]
public class OrderDetails
{
    public int Id { get; set; }
    public string ShippingAddress { get; set; }
}

What’s new in EF 6.3

We understand that many existing applications use previous versions of EF, and that porting them to EF Core only to take advantage of .NET Core can require a significant effort. For that reason, we decided to port the newest version of EF 6 to run on .NET Core 3.0. The developer community also contributed to this release with several bug fixes and enhancements.

Here are some of the most notable improvements:

  • Support for .NET Core 3.0
    • The EF 6.3 runtime package now targets .NET Standard 2.1 in addition to .NET Framework 4.0 and 4.5.
    • This means that EF 6.3 is cross-platform and supported on other operating systems besides Windows, like Linux and macOS.
    • The migration commands have been rewritten to execute out of process and work with SDK-style projects.
  • Support for SQL Server hierarchyid
  • Improved compatibility with Roslyn and NuGet PackageReference
  • Added the ef6.exe utility for enabling, adding, scripting, and applying migrations from assemblies. This replaces migrate.exe

There are certain limitations when using EF 6.3 in .NET Core. For example:

  • Data providers need to be also ported to .NET Core. We only ported the SQL Server provider, which is included in the EF 6.3 package.
  • Spatial support won’t be enabled with SQL Server because the spatial types aren’t enabled to work with .NET Core.
    • Note that this limitation applies to EF 6.3 but not to EF Core 3.0. The latter continues to support spatial using the NetTopologySuite library.
  • There’s currently no support for using the EF designer directly on .NET Core or .NET Standard projects.

For more details on the EF 6.3 release, and a workaround to the latter limitation, see What’s new in EF 6.3 in the product’s documentation.

What’s next: EF Core 3.1

The EF team is now focused on the EF Core 3.1 release, which is planned for later this year, and on making sure that the documentation for EF Core 3.0 is complete.

EF Core 3.1 will be a long-term support (LTS) release, which means it will be supported for at least 3 years. Hence the focus is on stabilizing and fixing bugs rather than adding new features and risky changes. We recommend that you adopt .NET Core 3.0 today and then adopt 3.1 when it becomes available. There won’t be breaking changes between these two releases.

The full set of issues fixed in 3.1 can be seen in our issue tracker. Here are some worth mentioning:

  • Fixes and improvements for issues recently found in the Cosmos DB provider
  • Fixes and improvements for issues recently found in the new LINQ implementation
  • Lots of regressions tests added for issues verified as fixed in 3.0
  • Test stability improvements
  • Code cleanup

The first preview of EF Core 3.1 will be available very soon.

Thank you

If you either sent code contributions or feedback for any of our preview releases, thanks a lot! You helped make EF Core 3.0 and EF 6.3 significantly better!

We hope everyone will now enjoy the results.

57 comments

Comments are closed. Login to edit/delete your existing comments

  • Avatar
    Albert Bielecki

    This is great! EF 6.3 should make moving to Core a lot easier. Great work guys!

  • Raul Rodriguez
    Raul Rodriguez

    At my company we would love to use EF Core, but we keep using EF 6.3 as it is unusable as today. First we waited for the solution to the “group by” commands being executed in memory (incredible!!). Every time we think we could finally use it, we find some new showstopper. This time we will not even try.

    We read the following post Migrating to .NET Core 3.0 and the opinion of the author is 100% our experience.

    Don’t get me wrong. As I said, we would really love to use it and I think the current team seems to be taking the right decisions (queries should work at the server and if not, it should be explicit). We have the feeling, that some decisions from the past were completely wrong and now you are trying to work around it. It is just that it seems to take too long and it is too fragile.

    I feel with you, as every developer has some examples of such code bases being difficult or impossible to get right.

    Keep trying as a working EF Core is essential for .NET. Thank you for the hard work.

    • Avatar
      James Hancock

      Agreed. EF Core 3.x is WORSE in every way than EF Core 2.2. (Yes we had throw as error already set for items that would be evaluated in memory)

      So many queries just don’t work anymore. Ones that are obviously and simply mapped to a single SQL query don’t work. Bugs that provide useless information that makes it impossible to identify what’s going on.

      Null Reference errors that are never supposed to happen but do, but only in the case where it’s evaluating the last leg of the query in memory based on the results it got back. (and you’ll never find them other than commenting out parts of the select until you find the one it’s tantruming about)

      The In Memory provider will error on queries that work just fine with exactly the same data against a real database that has nothing to do with relational commands. They just fail with null errors about bindings.

      And no, .NET 5 hasn’t improved this at all thus far. We’ve had 3 rewrites of EF Core and it’s still a steaming pile that is the #1 source of issues and wastes of time for our developers. If our code base wasn’t so large, we would have dumped EF entirely by now. As it is, it may still be on the table since the EF team is so unresponsive and so committed to reinventing the wheel instead of moving the product forward. (They won’t even fix bugs between versions for fear of breaking other stuff the new version is still so brittle!)

  • Avatar
    Jason Rosenthal

    Shouldn’t Microsoft pull the plug on EF Core already? I’m a database dev targeting Windows mainly and I always get burned trying to get anywhere with EF Core. I also like a database first option.

    • Avatar
      Ian Marteens

      Totally agree. The list of things to fix for 3.1 is awesome. The list of missing things also is large. And, yes: database first is the way to go. I’m a SQL guy: no cheap substitutes, please.

    • Avatar
      James Hancock

      The list of things that don’t work, work inconsistently or are just outright useless is surreal. It has gotten WORSE since .NET Core 2.2, not better (yes we had throw as error on client side evaluation in .NET Core 2.2 so that’s not why!)

      Meanwhile, instead of adding things like Delete by where, delete by id, patch support, better left outer join support etc. like they should have been adding they still haven’t even gotten close to what EF 6.3 had 5 years ago.

      Pull the plug, this sucker has flatlined.

  • Avatar
    Manuel Haas

    “Single SQL statement per LINQ query” in combination with possible runtime exceptions is an absolutely show stopper for us.
    In our App we have currently about 700 queries which contains the “ThenInclude” keyword. Often queries are created programmatically depending on program flow.
    All of this queries must be tested (and rewritten) with all possible program flows!!!
    Some queries loading data with 30 or more .Include, .ThenIncludes into objects. All this must be restructured…

    Does Microsoft use EF Core for a productive in-house application?
    I don’t think, otherwise the EF Core team would feel a significant headwind with these Breaking Changes.

    With “production-ready” software, you can’t afford that.

    • Avatar
      Jon Miller

      I haven’t tried it, but, it sounds like they reverted the behavior to what it was in EF 6. I.e. one big query. I have found that it works better for some things, but, maybe not others. It would be nice if there was a way to control the behavior. However, I am using Web Forms, so, I can’t use the new EF Core anyways now that it requires .NET Standard 2.1.

  • Jérémy Halldén
    Jérémy Halldén

    Does EF 6.3 run on .net 4.7?
    The dependencies section of nuget lists .net standard 2.1 which is not supported by 4.x, but from what I can see none of the listed dependencies need a higher version than .net standard 2.0 – so why is 2.1 stated?
    Have I misunderstood something?
    Any clarification on this would be helpful 🙂

    • Avatar
      Jérémy Halldén

      @divegamicrosoft-com (or anyone else)
      I would really appreciate a response to this question.

  • Mark Struik
    Mark Struik

    Hi, thanks for the nice update,

    This might be a silly question: will EF Core 2 be supported by .Net Core 3? as I am stuck with an oracle provider that will not be updated yet, but I do want to move forward with .Net Core 3.

    kind regards,

    Mark

    • Guy Benhaim
      Guy Benhaim

      Hello,
      With EDMX migration code I fail in joining two tables: using Method syntax the properties of the 2nd table do not appear and prevent defining the InnerKeySelctor (Only the External Declaration options appear).
      This causes an error and I cannot compile.

      Example:
      var list = db.PP.Where(pp => pp.F0 == 1).Join(db.OP, pp => pp.F1, op => op.XXXX , (pp, op) => new {pp, op}).Select(pp => pp.F3_ID).ToList();

      The issue: for op the properties are NOT shown by VS, and therefore we cannot use any property for the key.

      Surprisingly, using Query syntax the properties appear and the join is performed ok.

      Please advise.

  • Kinshuk Choubisa
    Kinshuk Choubisa

    Hello Vega,

    How to use sql IN query in EF core ? i don’t want to use contents.