In this article we will see how to pre-compile a query, its limitations and how it really improves the speed of a Linq query.

To pre-compile a query we must define a public static field of type Func<T,S,R> . What we assign to this field is the result of CompiledQuery.Compile method:

public static Func<testDataContext , SearchCriteria, IQueryable<Person>> FilteredResult …

In the above line, testDataContex is the type of a DataContext inside the project, SearchCriteria is type of a class or struct that is designed for passing search criteria to .Compile method. For example, suppose that in testDataContext, we have a Table named Person. We have also defined a class (or struct) named SearchCriteria as bellow:

public class SearchCriteria
{
public int id { set; get; }
public string FirstName { set; get; }
public string LastName { set; get; }
}

Now to get these definitions to work with a precompiled query we can write such a statement:

public static Func<testDataContext , SearchCriteria, IQueryable<Person>> FilteredResult =
System.Data.Linq.CompiledQuery.Compile(
(testDataContext dc , SearchCriteria criteria ) =>
from p in dc.Persons
where (p.id == criteria.id || criteria.id == -1)
&& (p.FirstName == criteria.FirstName || criteria.FirstName == string.Empty)
&& (p.LastName == criteria.LastName || criteria.LastName == string.Empty)
select p
);

That’s it. At this point, FilteredResult contains a pre-compiled query and can be used this way:

testDataContext dc = new testDataContext();
SearchCriteria criteria = new SearchCriteria();
criteria.id = -1;
criteria.FirstName = “Bill”;
criteria.LastName = “Gates”;
List<Person> p = FilteredResult(dc, criteria).ToList();

The above code creates instances of testDataContext (dc) and SearchCriteria (criteria) and passes them to FilteredResult as arguments. The result of FilteredResult is IQueryable<Person> we have called .ToList() extension method to get a List<Person> series.

One upsetting point about pre-compiled queries is that you can not use a stored-procedure to make a compiled query. In the above Linq to Sql code, if you write “from C in usp_GetPerson() …” you will get an error indicating that stored procedures are not allowed to be used.

Now let’s see how much precompilation can be helpful. I have written a small Console application that runs two version (one is compiled and one is not) of a query over a database for 1000 times. The time needed to run each query is as follows:

Compiled query takes 0 minutes, 1 seconds and 62 milliseconds.

Regular query takes 0 minutes, 13 seconds and 328 milliseconds.

As it is clear, the compiled query is greatly faster than a regular query. Notice that in a Linq model, nothing will really happen unless we iterate over the result of the query. Therefore, I have written a foreach statement to iterate over the result of queries. I also have written a small query at the beginning of the program to make Linq manager open a connection to Sql Server. If we do not do this, the compiled query will surprisingly takes longer!

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Data.Linq;

namespace CompiledQuery
{
class Program
{
public static Func<testDataContext , SearchCriteria, IQueryable<Person>> FilteredResult =
System.Data.Linq.CompiledQuery.Compile(
(testDataContext dc , SearchCriteria criteria ) =>
from p in dc.Persons
where (p.id == criteria.id || criteria.id == -1)
&& (p.FirstName == criteria.FirstName || criteria.FirstName == string.Empty)
&& (p.LastName == criteria.LastName || criteria.LastName == string.Empty)
select p
);

static void Main(string[] args)
{
testDataContext dc = new testDataContext();
SearchCriteria criteria = new SearchCriteria();
IQueryable<Person> Q = null;

// The following code makes Linq manager to open a connection to Sql Server
var init = from p in dc.Persons select p;
foreach (Person person in init) ;

criteria.id = -1;
criteria.FirstName = “Bill”;
criteria.LastName = “Gates”;
DateTime BeginTime = DateTime.Now;

for (int i = 0; i < 1000; i++)
{
Q = FilteredResult(dc, criteria);
foreach (Person person in Q) ;
}

DateTime EndTime = DateTime.Now;
TimeSpan Diff1 = EndTime – BeginTime;

BeginTime = DateTime.Now;

for (int i = 0; i < 1000; i++)
{
Q = from p in dc.Persons
where (p.id == criteria.id || criteria.id == -1)
&& (p.FirstName == criteria.FirstName || criteria.FirstName == string.Empty)
&& (p.LastName == criteria.LastName || criteria.LastName == string.Empty)
select p;
foreach (Person person in Q) ;
}

EndTime = DateTime.Now;
TimeSpan Diff2 = EndTime – BeginTime;

Console.WriteLine(“Compiled query takes : {0}:{1}:{2}”, Diff1.Minutes, Diff1.Seconds, Diff1.Milliseconds);
Console.WriteLine(“Regular query takes {0}:{1}:{2}”, Diff2.Minutes, Diff2.Seconds, Diff2.Milliseconds);

Console.ReadKey();
}
}
}

You can download the source of of a full sample project from here.

——————————————————————————————————————————————

In the earlier posts on LINQ to SQL performance optimization, I discussed about the LINQ Best Practices and LINQ DataLoadOptions. This discussion will remain incomplete if I don’t included CompiledQuery.Compile.

While developing the application we often come across scenario where we are required to execute a similar nature of query more than couple of times.

When we execute a query using LINQ there are number of steps involved before final SQL query is generated, and each time we call the method to fetch the data whole process starting from query  generation to its execution is carried out. In this life cycle the overhead that is easily visible is that we are every time generating the query.

LINQ’s CompiledQuery.Compile is used to avoid this extra overhead. If you have a application executing structurally similar query, CompiledQuery.Compile is one thing that can help you increase the performance of your application.

CompiledQuery.Compile compiles the query once and execute it for the rest of the time with different set of parameters.

// Method in which whole process of query generation and execution take place everytime

private static void CompiledQueryMethod(string orderID)

{

SampleDataContextDataContext db = new SampleDataContextDataContext();

var result = from c in db.Customers

join o in db.Orders on c.CustomerID equals o.CustomerID

where o.OrderID == orderID

select new { c };

}

In this case whole process of query generation to execution is carried out each time CompiledQueryMethod is called.

Now lets do the same using CompiledQuery.Compile way.

// This will compile the query and return a delegate that can be used to execute for diffferent

// set of parameters

public static Func<SampleDataContextDataContext, int, IQueryable<Customer>>

GenerateCompiledQuery = CompiledQuery.Compile((SampleDataContextDataContext db, int orderID) =>

from c in db.Customers

join o in db.Orders on c.CustomerID equals o.CustomerID

where o.OrderID == orderID

select c);

The above method returns a delegate that can be cached and used several number of time just by changing the input parameter.

private static void CompiledQueryMethod(int orderID)

{

SampleDataContextDataContext db = new SampleDataContextDataContext();

var result = GenerateCompiledQuery(db,orderID);

}

1. Type safety: I think we all understand this.
2. Abstraction: This is especially true with LINQ-to-Entities. This abstraction also allows the framework to add additional improvements that you can easily take advantage of. PLINQ is an example of adding multi-threading support to LINQ. Code changes are minimal to add this support. It would be MUCH harder to do this data access code that simply calls sprocs.
3. Debugging support: I can use any .NET debugger to debug the queries. With sprocs, you cannot easily debug the SQL and that experience is largely tied to your database vendor (MS SQL Server provides a query analyzer, but often that isn’t enough).
4. Vendor agnostic: LINQ works with lots of databases and the number of supported databases will only increase. Sprocs are not always portable between databases, either because of varying syntax or feature support (if the database supports sprocs at all).
5. Deployment: Others have mentioned this already, but it’s easier to deploy a single assembly than to deploy a set of sprocs. This also ties in with #4.
6. Easier: You don’t have to learn T-SQL to do data access, nor do you have to learn the data access API (e.g. ADO.NET) necessary for calling the sprocs. This is related to #3 and #4.

Some disadvantages of LINQ vs sprocs:

1. Network traffic: sprocs need only serialize sproc-name and argument data over the wire while LINQ sends the entire query. This can get really bad if the queries are very complex. However, LINQ’s abstraction allows Microsoft to improve this over time.
2. Less flexible: Sprocs can take full advantage of a database’s featureset. LINQ tends to be more generic in it’s support. This is common in any kind of language abstraction (e.g. C# vs assembler).
3. Recompiling: If you need to make changes to the way you do data access, you need to recompile, version, and redeploy your assembly. Sprocs can sometimes allow a DBA to tune the data access routine without a need to redeploy anything.

Security and manageability are something that people argue about too.

1. Security: For example, you can protect your sensitive data by restricting access to the tables directly, and put ACLs on the sprocs. With LINQ, however, you can still restrict direct access to tables and instead put ACLs on updatable table views to achieve a similar end (assuming your database supports updatable views).
2. Manageability: Using views also gives you the advantage of shielding your application non-breaking from schema changes (like table normalization). You can update the view without requiring your data access code to change.

OracleParameter outParam = new OracleParameter(“p_NUM_RECS_AFFECTED”, OracleDbType.Decimal, ParameterDirection.Output);
oRequest.Parameters.Add(outParam);

ORADataFactory oFactory = new ORADataFactory();
oFactory.ExecuteNonQuery(oRequest);

int numRecsAffected = 0;
int.TryParse(outParam.Value.ToString(), out numRecsAffected);

return numRecsAffected;
}

————————-Ref Cursor ——————————
DECLARE
type result_set is ref cursor;
BEGIN
SK_CORAL.STAT_CERTIFICATE_PKG.GENERATENONNAICSUSPANCE(’31-OCT-2010′,:result_set);
END;

Although orignally developed for aviation, WAAS is not just limited to the aviation industry, any GPS receiver that is capable of receiving the WAAS signal will be able to benefit from it’s correction data, making the GPS positioning more accurate. In fact a WAAS-enabled GPS receiver can even give you directions right down to the lane your car is traveling in (as long as the maping program supports “lane assist” directions). Because the Wide Area Augmentation System is quickly becoming standard in the GPS industry, most new GPS receivers today are WAAS-enabled. Just like with the conventinal GPS, the WAAS system doesn’t come with any extra cost or fees to use. All that is required is that the GPS receiver be WAAS-enabled so it can receive and decode the data then be able to apply corrections to it’s position. Currently the WAAS service is limited to the U.S.A., Canada, Alaska and Hawaii. Although independant from WAAS, Europe and Asia are working on their own supplemental GPS correction systems. Europe has the “Euro Geostationary Navigation Overlay Service” (EGNOS) and Japan is working on their “Multi-Functional Satellite Augmentation System” (MSAS) .

How The WAAS System Works

WAAS uses a network of approximately 25 ground based Wide-area Reference Stations (WRS) in North America and Hawaii, to measure small variations in GPS satellite signals in the western hemisphere. These precisely surveyed ground stations monitor and collect information on the GPS signals and send their data to the three Wide-area Master Stations (WMS). The WMS’s generate two different sets of corrections: fast and slow. The fast corrections are for errors that are changing rapidly and are a primary concern to the GPS satellites instantaneous positions and clock errors. These corrections are user position independent, which means they can be applied instantly by any receiver in the WAAS broadcasting area. The slow corrections are for long-term ephemeric and clock error estimates and ionospheric delay information.

Once these corrections are generated, the Master Stations sends them to two pairs of Ground Uplink Stations (GUS) that transmit the correction messages to a series of geostationary satellites that broadcast their correction data back to earth. Then WAAS-enabled GPS receivers use this information to make corrections to the original GPS signial, giving WAAS-enabled GPS receiver a more accurate position. GPS receiver’s use the information broadcast from each GPS satellite to determine their location and the current time. Depending on the GPS device, a GPS receiver only needs to receive a signal from 3-4 satellites (out of the 31 satellites currently transmiting a signal for civilan users) to be able to calculate it’s position. In addition to the GPS signal, a WAAS-enabled GPS receiver can also receive the geostationary WAAS satellite signal.

The two different types of correction messages from the WAAS system (fast and slow) are used by the GPS receiver in different ways. The fast type of correction data includes the corrected satellite position and clock data to determine its current location using normal GPS calculations. Once an approximate position fix is obtained the GPS receiver begins to use the slow corrections to improve its accuracy. Slow correction data Includes the ionospheric delay. When the GPS signal travels from the satellite to the receiver, it passes through the ionosphere. The receiver calculates the location where the signal pierced the ionosphere and, if it has received an ionospheric delay value for that location, it corrects for the error that the ionosphere created. Unlike the fast data, the slow data doesn’t need to be updated frequently because the ionosphere conditions don’t change rapidly. While the slow data can be updated every minute if necessary, they are only updated every two minutes and are considered valid for up to six minutes.

Limitations of the WAAS System

(1) The WAAS system is currently only available to United States and large parts of Canada and Alaska but there plans to expand the system to other countries and continents.

(2) Because the WAAS broadcasting satellites are geostationary causes them to be less than 10° above the horizon for locations north of 71.4° latitude. This means aircraft in areas of Alaska or northern Canada may have difficulty maintaining a lock on the WAAS signal.

(3) In order to calculate an ionospheric grid point’s delay, that point must be located between a satellite and a reference station. The low number of satellites and ground stations limit the number of points which can be calculated.

(4) Aircraft conducting WAAS approaches must possess certified GPS receivers.

ref_cursors

Strongly Typed
Note: A REF CURSOR that specifies a specific return type.

Package Header CREATE OR REPLACE PACKAGE strongly_typed IS

TYPE return_cur IS REF CURSOR RETURN all_tables%ROWTYPE;
PROCEDURE child(p_return_rec OUT return_cur);
PROCEDURE parent(p_NumRecs PLS_INTEGER);

END strongly_typed;
/

Package Body CREATE OR REPLACE PACKAGE BODY strongly_typed IS
PROCEDURE child(p_return_rec OUT return_cur) IS

BEGIN
OPEN p_return_rec FOR
SELECT * FROM all_tables;
END child;
–==================================================
PROCEDURE parent (p_NumRecs PLS_INTEGER) IS
p_retcur return_cur;
at_rec all_tables%ROWTYPE;
BEGIN
child(p_retcur);

FOR i IN 1 .. p_NumRecs
LOOP
FETCH p_retcur
INTO at_rec;

dbms_output.put_line(at_rec.table_name ||
‘ – ‘ || at_rec.tablespace_name ||
‘ – ‘ || TO_CHAR(at_rec.initial_extent) ||
‘ – ‘ || TO_CHAR(at_rec.next_extent));
END LOOP;
END parent;
END strongly_typed;
/
To Run The Demo set serveroutput on

exec strongly_typed.parent(1)
exec strongly_typed.parent(8)

Weakly Typed
Note: A REF CURSOR that does not specify the return type such as SYS_REFCURSOR.

Child Procedure CREATE OR REPLACE PROCEDURE child (
p_NumRecs IN PLS_INTEGER,
p_return_cur OUT SYS_REFCURSOR)
IS

BEGIN
OPEN p_return_cur FOR
‘SELECT * FROM all_tables WHERE rownum <= ' || p_NumRecs ;
END child;
/

Parent Procedure CREATE OR REPLACE PROCEDURE parent (pNumRecs VARCHAR2) IS
p_retcur SYS_REFCURSOR;
at_rec all_tables%ROWTYPE;
BEGIN
child(pNumRecs, p_retcur);

FOR i IN 1 .. pNumRecs
LOOP
FETCH p_retcur
INTO at_rec;

dbms_output.put_line(at_rec.table_name ||
' – ' || at_rec.tablespace_name ||
' – ' || TO_CHAR(at_rec.initial_extent) ||
' – ' || TO_CHAR(at_rec.next_extent));
END LOOP;
END parent;
/
To Run The Demo set serveroutput on

exec parent(1)
exec parent(17)

Passing Ref Cursors

Ref Cursor Passing Demo CREATE TABLE employees (
empid NUMBER(5),
empname VARCHAR2(30));

INSERT INTO employees (empid, empname) VALUES (1, 'Dan Morgan');
INSERT INTO employees (empid, empname) VALUES (2, 'Hans Forbrich');
INSERT INTO employees (empid, empname) VALUES (3, 'Caleb Small');
COMMIT;
CREATE OR REPLACE PROCEDURE pass_ref_cur(p_cursor SYS_REFCURSOR) IS

TYPE array_t IS TABLE OF VARCHAR2(4000)
INDEX BY BINARY_INTEGER;

rec_array array_t;

BEGIN
FETCH p_cursor BULK COLLECT INTO rec_array;

FOR i IN rec_array.FIRST .. rec_array.LAST
LOOP
dbms_output.put_line(rec_array(i));
END LOOP;
END pass_ref_cur;
/

set serveroutput on

DECLARE
rec_array SYS_REFCURSOR;
BEGIN
OPEN rec_array FOR
'SELECT empname FROM employees';

pass_ref_cur(rec_array);
CLOSE rec_array;
END;
/

Introduction to REF CURSOR

A REF CURSOR is basically a data type. A variable created based on such a data type is generally called a cursor variable. A cursor variable can be associated with different queries at run-time. The primary advantage of using cursor variables is their capability to pass result sets between sub programs (like stored procedures, functions, packages etc.).

Let us start with a small sub-program as follows:

declare
type r_cursor is REF CURSOR;
c_emp r_cursor;
en emp.ename%type;
begin
open c_emp for select ename from emp;
loop
fetch c_emp into en;
exit when c_emp%notfound;
dbms_output.put_line(en);
end loop;
close c_emp;
end;

Let me explain step by step. The following is the first statement you need to understand:

type r_cursor is REF CURSOR;

The above statement simply defines a new data type called "r_cursor," which is of the type REF CURSOR. We declare a cursor variable named "c_emp" based on the type "r_cursor" as follows:

c_emp r_cursor;

Every cursor variable must be opened with an associated SELECT statement as follows:

open c_emp for select ename from emp;

To retrieve each row of information from the cursor, I used a loop together with a FETCH statement as follows:

loop
fetch c_emp into en;
exit when c_emp%notfound;
dbms_output.put_line(en);
end loop;

I finally closed the cursor using the following statement:

close c_emp;

%ROWTYPE with REF CURSOR

In the previous section, I retrieved only one column (ename) of information using REF CURSOR. Now I would like to retrieve more than one column (or entire row) of information using the same. Let us consider the following example:

declare
type r_cursor is REF CURSOR;
c_emp r_cursor;
er emp%rowtype;
begin
open c_emp for select * from emp;
loop
fetch c_emp into er;
exit when c_emp%notfound;
dbms_output.put_line(er.ename || ' – ' || er.sal);
end loop;
close c_emp;
end;

In the above example, the only crucial declaration is the following:

er emp%rowtype;

The above declares a variable named "er," which can hold an entire row from the "emp" table. To retrieve the values (of each column) from that variable, we use the dot notation as follows:

dbms_output.put_line(er.ename || ' – ' || er.sal);

Let us consider that a table contains forty columns and I would like to retrieve fifteen columns. In such scenarios, it is a bad idea to retrieve all forty columns of information. At the same time, declaring and working with fifteen variables would be bit clumsy. The next section will explain how to solve such issues.

MERGE INTO table
USING table
ON ()
WHEN MATCHED THEN
DELETE
WHEN NOT MATCHED THEN
[LOG ERRORS <reject limit ];

CREATE TABLE employee (
employee_id NUMBER(5),
first_name VARCHAR2(20),
last_name VARCHAR2(20),
dept_no NUMBER(2),
salary NUMBER(10));

INSERT INTO employee VALUES (1, ‘Dan’, ‘Morgan’, 10, 100000);
INSERT INTO employee VALUES (2, ‘Helen’, ‘Lofstrom’, 20, 100000);
INSERT INTO employee VALUES (3, ‘Akiko’, ‘Toyota’, 20, 50000);
INSERT INTO employee VALUES (4, ‘Jackie’, ‘Stough’, 20, 40000);
INSERT INTO employee VALUES (5, ‘Richard’, ‘Foote’, 20, 70000);
INSERT INTO employee VALUES (6, ‘Joe’, ‘Johnson’, 20, 30000);
INSERT INTO employee VALUES (7, ‘Clark’, ‘Urling’, 20, 90000);

CREATE TABLE bonuses (
employee_id NUMBER, bonus NUMBER DEFAULT 100);

INSERT INTO bonuses (employee_id) VALUES (1);
INSERT INTO bonuses (employee_id) VALUES (2);
INSERT INTO bonuses (employee_id) VALUES (4);
INSERT INTO bonuses (employee_id) VALUES (6);
INSERT INTO bonuses (employee_id) VALUES (7);
COMMIT;

SELECT * FROM employee;

SELECT * FROM bonuses;

MERGE INTO bonuses b
USING (
SELECT employee_id, salary, dept_no
FROM employee
WHERE dept_no =20) e
ON (b.employee_id = e.employee_id)
WHEN MATCHED THEN
UPDATE SET b.bonus = e.salary * 0.1
DELETE WHERE (e.salary 40000);