Azure IoT Solution Architecture Best Practices (Common Recommendations – part 1)

When we are talking on successful design of IoT solutions there are two different aspects:

  • Common (technology agnostic) principles and recommendation.
  • Knowledge on a specific platform, which provides components to build IoT solution (like Microsoft Azure)

Considering work on specific solutions we need first to know the common principles and afterwards to has a knowledge on a specific platform, used for implementation.

  • Reduce the Complexity:

One of the biggest challenges you face when you are planning Internet of Things (IoT) solutions is dealing with complexity.
This is a common principle for all solutions – not only for IoT. IoT solutions involves many heterogeneous IoT devices, with sensors that generate data that is then analyzed to gain insights. IoT devices are connected either directly to a network or through a gateway device to a network, communicating with each other and with cloud services and applications.

We need to strategies strategies which help us to simplify development, manage complexity, and ensure that your IoT solutions remain scalable, flexible, and robust:

  • Assume a layered architecture

An architecture describes the structure of your IoT solution, including the physical aspects (that is, the things) and the virtual aspects (like services and communication protocols). Adopting a multi-tiered architecture allows you to focus on improving your understanding about how all of the most important aspects of the architecture operate independently before you integrate them within your IoT application. This modular approach helps to manage the complexity of IoT solutions.
The main question is what is the best design regarding the layered architecture:
The good design can be done in accordance with functional and non-functional requirements:
Considering data-driven IoT applications that involve edge analytics, a basic three-tiered architecture, captures the flow of information from devices, to edge services, and then out to cloud services. A detailed IoT architecture can also include vertical layers that cut across the other layers, like identity management or data security.

Layered-IoT-Architecture
There are IoT solutions based only on two main layers: devices and back-end services (usually cloud based). Solutions, which to not require preliminary processing or prompt response after real time analysis can be based on  such kind of architecture.

  • Implement “Security by Design”

Security must be a priority across all of the layers of your IoT architecture. We need to think about security as a cross-cutting concern in your IoT architecture, rather than as a separate layer of your IoT architecture. With so many devices connected, the integrity of the system as a whole needs to be maintained even when individual devices or gateways are compromised. We need to have a security in every module, every layer and for the overall IoT Solution:

We need to adopt standards and best practices for these aspects of your IoT infrastructure:

  1. Device, application and user identity, authentication, authorization, and access control
  2. Key management
  3. Data security
  4. Secure communication channels and message integrity (by using encryption)
  5. Auditing
  6. Secure development and delivery

 

  • Data Model

IoT solutions are data-centric. Each IoT application has a data model that includes data from the devices, as well as user generated data and data from outside systems. The data model must support the use cases in accordance with the solution design / customer requirements. 
Most of IoT systems need to have more than one data storage where we need:

 

  1. Raw data storage (blob, file, noSQL)
  2. Metadata storage (SQL, optional noSQL)
  3. Aggregated data storage (SQL, noSQL)
  4. Configuration data storage (SQL, optional noSQL)

 

  • Raw Data Storage

Raw data storage is fundamental for IoT Solutions, where one of the main functionality is to ingress information from sensors. Usually we have no need of 
schema for these messages. Message format could be updated without any changes in DB structure.
One of the main requirement for most of solutions is this storage to be scalable (partitioned). Relational (SQL) databases support not so very good partitioning and from another hand we need to be aligned to a specific database schema. SQL databases are also most expensive than noSQL solutions or binary/blob storages.

We have several often used options to store the raw data:

  1. Blob storage or files (file storage)
  2. Cheap noSQL storage (like Key Value databases)
  3. Document oriented databases
  4. TIme series databases

Blob storage or files. There are many reasons to consider Binary Large Objects (BLOBs), or what are more commonly known as files. The storage is cheap and easy to use. The main disadvantage is that we need to consider additional service or custom implementation to search specific information using this storage.

Key-value stores save data as associative arrays where a single value is associated together with a key used as a signifier for the value. Cheap options lioke Azure Table Storage offer limited options for searching (only on the main key and partition key). Others (like Redis or CosmosDB/Table API) allow multi indexing, but on a higher price.

 

Different options for cheap storage in Microsoft Azure.

 

Document oriented databases offer support to store semi-structured data.  It can be JSON, XML, YAML, or even a Word Document.  The unit of data is called a document (similar to a row in RDBMS).  The table which contains a group of documents is called as a “Collection”.

 

 

Document oriented database design

TIme series databases: TSDBs are databases that are optimized for time series data. Software with complex logic or business rules and high transaction volume for time series data may not be practical with traditional relational database management systems. It is possible these solutions to be based on blob storage or document databases, but with additional logic, which allows users to create, enumerate, update and destroy various time series and organize them in some fashion.

Time series database

  • Metadata storage

Metadata storage usually contains additional information about main entities, which we have in our system (sensors and other devices, users and other abstractions, related to the applied area, for which will be used the IoT system), as well as the relations between these entities. Usually this information is not very big and it is easier to store it in SQL database.  There are solutions where metadata is stored in document oriented database, but usually the simplest approach is to design metadata model in RDBMS.

  • Aggregated data storage

The collected data is being processed (often in real time ) where row data is enriched and aggregated. Aggregated data is used for reports on specific period of time bases. Aggregated data is not so bug as a row data and usually is stored in SQL database or in document oriented database.

  • Configuration data storage.

All setting for the IoT system need to be stored in a storage. Most often this is SQL database, because data is small, but RDBMS support rich and easy to design schema. It is possible to use also noSQL for configuration settings.

 

If you want more information feel free to contact me at michael@mateev.net Follow my blog : https://mmateev.wordpress.com/  .

You can learn more also if you follow me on Twitter @mihailmateev  , and stay in touch on Facebook, Google+, LinkedIn and Bulgarian BI and .Net User Group !

Advertisements
Posted in Azure, Azure Blob Storage, Azure DocumentDB, Azure Storage, Cloud, Internet of Things, IoT, Security, Solution Architecture, SQL | Leave a comment

Azure IoT Solution Architecture Best Practices (Introduction)

Internet of Things (IoT) is one of the areas in IT industry which is growing rapidly last several years and will continue to grow next years. It is expected from 2015 to 2025 the number of connected devices to grow 5 times and to have 75 billion connected devices in 2025 (15 billion in 2015) regarding Statista.

IoT-Devices-2025

In a series of blogs will be shared experience in design of IoT Solutions (best practices). Most of the content will be related to solutions, based on Microsoft Azure.

IoT Basics:

The modern IoT solutions could be very complex and need expertise in different areas:

  • Hardware
  • Networking
    / connectivity

  • Solution design
  • Application development
  • Security
  • Business intelligence and data analytics
  • Machine learning and artificial intelligence (AI)

1. Hardware:
In the heart of IoT are the billions of interconnected “things,” or devices with attached sensors and actuators that sense and control the physical world. Now technology offers many imbedded devices (many of them quite powerful) , “smart” devices, used in different industries: automotive, energy area, healthcare, logistics, wearables etc.

2. Networking/ Connectivity:
The is another one of the key aspect of IoT. Growing of internet and global connectivity allows much easier to connect embedded devices with back end servers worldwide. Now there are much less limitations, related to connectivity between devices and server applications, but network design is a major part of the solution architecture of each Internet of Things system.
In addition to network design, developers should have a working knowledge of network standards, protocols, and technologies. These include wifi, Low Energy Bluetooth, Zigbee, cellular, and RFID technologies used in consumer applications, as well as Low Power Wide-Area Network (LPWAN) technologies like LoRa.

3. Security:
Security is one of the biggest concerns in IoT.  Communication between embedded devices and back-end servers via internet can give many options for vulnerability. Security must be built-in at every step of the design of the system.  Other critical issues that are closely related to security include data ethics, privacy and liability.
The biggest challenges in IoT security are:

  • The physical and network access to embedded devices and their data
  • Back-end security
  • Communication between embedded devices and the solution back-end

Most of the embedded devices are still not so powerful to be possible to encrypt/decrypt data if for communication is used some cryptography algorithm. This make these devices more vulnerable and it is need to put it behind firewall.
The approach how to establish the communication (part of the solution design)  is also very critical for the security of Internet of Things solutions. 

4. Business intelligence and data analytics:
As the number of IoT devices transmitting data increases, big data turns into really big data.  Design and implementation needs big time data management skills to securely, fast and reliably ingest, store, and query big amount of heterogeneous data collated from these devices. This data management need to allow scalability and extensibility.

Many IoT devices generate latency or time-sensitive data, so it is necessary to filter or discard irrelevant data. Key technologies and platforms for data analytics that IoT developers should develop skills in include Hadoop / HDInsight, Spark, and NoSQL databases like CosmosDB, MongoDB, IBM Cloudant.

5. Machine Learning and AI:
Machine learning and AI skills are the final must-have skill for IoT developers and architects. Intelligent big data analytics involves applying cognitive computing techniques drawn from data mining, modeling, statistics, machine learning, and AI. These techniques are not appropriate for real-time data processing, but it can be applied in real-time to sensor data streams for predictive analysis or to autonomously make decisions in response to incoming data and can also be applied to historical data to identify patterns or anomalies in the data.

6. Application design and development:
This series of articles, starting with the current one are actually focused on architecture and design of IoT Solutions. These two aspects are the most important for an solution. Design is critical to have properly working, reliable and well performing solution. Technology is having progress very fast and the solutions design should be actual for the next several years. Developers should keep track of emerging frameworks and developer kits that they can leverage for rapid prototyping, as well as IoT platforms that provide infrastructure and tools to help automate building, deploying, managing, and operating IoT applications.

Next several articles from this series will be focused on the best practices in IoT slutions design and implementation.

If you want more information feel free to contact me at michael@mateev.net Follow my blog : https://mmateev.wordpress.com/  .

You can learn more also if you follow me on Twitter @mihailmateev  , and stay in touch on Facebook, Google+, LinkedIn and Bulgarian BI and .Net User Group !

Posted in Uncategorized | Tagged , , , , , , , | Leave a comment

How to simulate IoT Gateway using VMware Workstation

The IoT systems are one of the most interesting software solutions during the last last few years.  These systems collects information from different sensors and send this information to the back-end. Data is being processed ( aggregations, different kind of analysis ). Usually these solutions has separate storages for:

  • raw data
  • processed data
  • configuration data

The application created for this article series is a simple IoT solution implemented with C# and contains the following parts (see the picture below):

  • Managed objects (sensors), that measure specific values like temperature, speed, displacements, etc.
  • Gateways: devices that collects information from sensors and communicates via
  • Back office – set of services and applications that are used to ingress, process and store data, manage and monitor the whole solution.

The simple prototype used for this article includes:

  • Gateway (.NET application)
  • Gateway service: a web service, implemented on .NET and hosted in Microsoft Azure, that communicates with the gateway and stores data to a cloud storage ( Azure SQL Database ) 
  • Gateway service: IoT Hub – this is the recommended messaging service for IoT solutions in Azure
  • Azure Stream Analytics
  • Azure SQL Database 
  • PowerBI Reports, that represent some statistic, based on the generated messages

Cloudant-tut1-pic13[1]

Quite often during the development process usage of real gateway ( embedded devices like Raspberry Pi 2 and 3, MinnowBoard Max, DragonBoard 410c ) leads to more complexity and technical challenges. That is the reason to consider usage of “Simulated gateway”.

Simulated gateways could be just a client application, adapted to work on the development machine or virtualized client device.

The first approach can lead to some differences between development environment and the real production environment: development machines are usually much more powerful and often it is not possible to have the 100% identical codebase as this one on the embedded devices because of some differences in the SDKs.

Virtualization of the embedded devices gives from one side possibility to run the same code, that developers will use in the real devices, but from another hand gives a flexibility, because software engineers are less dependent on additional hardware during the development phase.

There are different virtualization platforms: Hyper-V, VMware, Virtual Box etc. Which one is better to choose ?
Embedded devices have some specific features, so probably it is better to consider on which platform can virtualize these devices relative easy and have quite good performance.

If you need different VMs on your workstation it is not possible to use Hyper V at the same time when you use other virtualization platforms. When you enable Hyper-V it changes the way windows on the host loads, it loads the hyper-v hypervisor and your host OS is actually running on top of that hypervisor

This is because the Hyper-V role is installed and this conflicts with VMware Workstation. To disable Hyper-V from starting the following command can be used:

  • bcdedit /set hypervisorlaunchtype off
  • A reboot of of the Windows OS is necessary.
  • To enable the Hyper-V role again use the following command:
  • bcdedit /set hypervisorlaunchtype auto
  • A reboot of of the Windows OS is necessary.

If you don’t want to change the settings any time you can create a multi-bootable configuration:

You could make a new boot entry. You can then choose on reboot whether to boot with Hyper V turned on or not.
This is useful as if Hyper V is installed then Virtual Box or VMware will not work. You get a message VT-x is not available (as Hyper-V is using it).
1. Install Hyper V
2. bcdedit /copy “{current}” /d “Hyper-V”
3. bcdedit /set “{current}” hypervisorlaunchtype off

More details you can find in this blog post:
Creating a “no hypervisor” boot entry – Ben Armstrong – Site Home – MSDN Blogs

How to setup Windows 10 IoT Core virtualized environment.

  • Downloads

You will also need a Windows 10 Core image. You can get one from Microsoft here. Choose Download Windows 10 IoT Core for MinnowBoard Max. This is important, because its easier to virtualize the x86 build. We now have to install it. Microsoft has bundled Windows 10 Core in an installer together with some tools for helping you provision it on a sd card and controlling it. So go ahead and run the downloaded installer. After the installation is complete you can find the image in C:\Program Files (x86)\Microsoft IoT\FFU\MinnowBoardMax

IoT-Gateway-VMWare-pic014[1]

  • Setting the virtual hard drive

You need to expand the ffu and create a virtual hard drive that VirtualBox can use. To do that we are going to use a 3rd party tool called ImgMount. it’s a community tool that can be found on xda developers here. What it does is it reads the ffu, converts it to a vhd virtual hard drive and mounts it. Download it and place it some place easy to call from the console. Now follow these steps:

  • Open a privileged powershell. Right click on powershell icon and click Run as administrator.
  • “cd” into the Windows 10 core image
  • Run the ImgMount tool providing the flash.ffu as an argument. You should see output similar to this:

IoT-Gateway-VMWare-pic015[1]

  • you have to unmount the image. Open Disk Management inside of Computer management. You will see a new disk that shows below your hard disks. To unmount it right click around the disk name and in the menu that opens select Detach VHD. A new dialog will open that will give you the location of the virtual hard disk that ImgMount has created. Now press OK in the dialog to unmount the virtual hard disk.
  • Navigate to the location of the virtual disk that we saved in notepad. After being unmounted it can be safely moved. Move it to the place you will going to store the virtual machine.

Using this approach it is possible to have a virtual disk in VHD format. There is additional work to run this VHD on Hyper-V. Hyper-V virtualization of Windows 10 Core devices is not a subject of this article.

  • Running Windows 10 IoT Core on VMware

Both VHD and VMDK contains hard disk image which used by virtual machines. Actually a VHD can be converted to VMDK format and used as the hard disk image for VMWare virtual machine. The virtual machines created in Microsoft virtualization products can be easily converted VMWare virtual machines with several pro0ducts like  VMware vCenter Converter and some free tools like StarWind V2V Converter .

StarWind V2V Converter is used in this example to convert the VHD file to VMware VMDK virtual disk.

  • In VMware Workstation you need need to create a new machine using a custom configuration.
  • The OS should be set to be installed later
  • The type of the OS is Windows 10
  • The firmware type should be EFI
  • The disk controller should be SATA
  • You need to use an existing disk adding the converted VMDX file.

That is all: you can run your device:

IoT-Gateway-VMWare-pic010[1]

New updates can be installed on the existing VM

IoT-Gateway-VMWare-pic011[1]

The graphical UI and command console are available in the same way like you connect an external monitor to the embedded device.

IoT-Gateway-VMWare-pic010a[1]

 IoT-Gateway-VMWare-pic011a[1]

All running devices are available in the IoT Dashboard.

IoT-Gateway-VMWare-pic012[1]

Next blogs on Windows IoT Core will demonstrate how you can develop and deploy a simple IoT solutions in Azure with much more details

If you want more information feel free to contact me at michael@mateev.net Follow my blog : https://mmateev.wordpress.com/  .

You can learn more also if you follow me on Twitter @mihailmateev  , and stay in touch on Facebook, Google+, LinkedIn and Bulgarian BI and .Net User Group !

Posted in .Net, Azure, BI, BI & .Net Geeks, BI & .Net UG, BI & .NET User Group, C#, IoT, IoT Suite, Microsoft, Microsoft Azure, Uncategorized, Windows 10, Windows 10 IoT Core | Tagged , , , , , | Leave a comment

SQLSaturday #432 CapeTown – Event Recap

I’m very glad that I attended as a  speaker at SQLSaturday #432 Cape Town. It was my first SQLSaturday in Cape Town, South Africa an in Africa in general. It was the first time when I presented on Azure Stream Analytics – the new SaaS complex event processing services offered from Microsoft for Microsoft Azure. I also presented for the first time session, 100% focused only on Entity Framework 7 – the new ORM from Microsoft, offering a new different approach using ORMs in the complex prokects.

I have shared the experience that we already have with my team in Stypes / ICT Automatisering using Stream Analytics as one of the data stores for iOTA ( Internet of Things Analytics ) – the IoT solution of ICT Automatisering / part of ICT Group.

SQLSAT432_SPEAKING 

The event was held on Saturday, September 12th  at the River Club – Cape Town observatorhy area. 

Administrator of the conference was Jody Roberts, Lead of the local SQL user group. Joddy is also a PASS Regional Mentor for MEA ( Middle East ansAfrica ) .  He is the most active community geek in the region, helping also in the organization of many other events in the region.  Jody organized this year SQLSaturday Cape Tiwn for the fifth time.

This was my first event in South Africa nad the third my event in MEA ( as a speaker at SQLSaturdays in Istanbul). Infragistics Inc. was the only one component vendor with a speaker at the conference. Participants gave a good feedback about my presentation.  There was an interest in the  IoT solutions and different options for data storages, related to these systems ( like Azure DocumentDB.

This event was also the biggest one SQLSaturday, ever organized in Lisbon

Summary:

  • There was 20 presentations in 4 tracks
  • Speakers from USA, UK, India, South Africa and Bulgaria 
  • nearly 200 attendees on site

Thanks to the whole SQLSaturday Lisbon team for the awesome organization and hospitality!

@ River Club, Cape Town

 12011258_10207586859700288_1513692064911969715_n[1]

11222261_10207586859660287_8550890657433130715_o[1]

11046262_10207588398058746_4637005614438463902_o[1]

If you want more information about the event and PASS community feel free to contact me at michael@mateev.net Follow this event on Twitter with hash tags  #sqlsatCapeTown and #sqlsat432.

You can learn more about the PASS events if you follow me on Twitter @mihailmateev  , and stay in touch on Facebook, Google+, LinkedIn and Bulgarian BI and .Net User Group !

Posted in Azure, Azure Storage, Azure Stream Analytics, Entity Framework 7, SQL, SQL Saturday, sqlfamily, sqlpass, sqlsat432, sqlsatCapeTown, Stream Analytics | Leave a comment

SQLSaturday #369 Lisbon Event Recap

I’m very glad that I attended as a  speaker at SQLSaturday #369 Lisbon . It was my third SQLSaturday in Lisbon and definitely the best ever one in Europe regarding to my experience. It was the first time when I presented on DocumentDB – the new SaaS noSQL database offered from Microsoft for Microsoft Azure.

I have shared the experience that we already have with my team in Stypes / ICT Automatisering using DocumentDB as one of the data stores for iOTA ( Internet of Things Analytics ) – the IoT solution of ICT Automatisering

11060255_10206675871366149_8918620088768012283_n[1]

The event was held on Saturday, May 16th  at Microsoft Portugal, Lisbon.

Administrator of the conference was  Niko Carvalho Neugebauer  Lead of SQLPort (SQL Server User Group) and BITuga ( TUGA – Business Intelligence User Group) of Portugal.  He organized an amazing team of volunteers who organized this awesome event.

This was the third my event in Portugal ( as a speaker at SQLSaturday Lisbon / Portugal ) and the sixth SQLSaturday in Portugal. Infragistics Inc. was the only one component vendor with a speaker at the conference. Participants gave a good feedback about my presentation.  There was an interest in the  IoT solutions and different options for data storages, related to these systems ( like Azure DocumentDB.

This event was also the biggest one SQLSaturday, ever organized in Lisbon

Summary:

  • There was 49 presentations in 7 tracks
  • More than 540 registrations
  • around 40 Speakers from many countries

Thanks to the whole SQLSaturday Lisbon team for the awesome organization and hospitality!

@ Microsoft Portugal, Lisbon

IMG_20150516_083526[1]

IMG_20150516_120531[1]

IMG_20150516_082453[1]

CFI1IMRUkAAQbbI[1]

IMG_20150516_185446[1]

If you want more information about the event and PASS community feel free to contact me at michael@mateev.net Follow this event on Twitter with hash tags  #sqlsatPortugal and #sqlsatLisbon.

You can learn more about the PASS events if you follow me on Twitter @mihailmateev  , and stay in touch on Facebook, Google+, LinkedIn and Bulgarian BI and .Net User Group !

Posted in .Net, Azure, Azure DocumentDB, Azure Storage, Azure Table Storage, BI, BI & .Net Geeks, DocumentDB, EF, IoT, NoSQL, SQL, SQL Saturday, SQL Server, sqlfamily, sqlpass, sqlsat369, sqlsatLisbon, sqlsatPortugal | Leave a comment

How to Manage Work Items Using Visual Studio Online REST API

TFS online as a part of Visual Studio Online ( VSO )  stays more and more popular solution for source control , backlog items management ( in the context of scrum and kanban ) , build system etc. Often software companies need to integrate work items management with other systems ( project management, billing etc.)

In the series of several blogs I will try to share my experience how to manage Visual Studio Programmatically.

In this article we will learn how to start to manage programmatically work items in Visual Studio Online. To do that you can use the VSO REST API calling it’s methods from any platform/language . Sometimes it is overhead to create own library to work with Visual Studio Online REST API. In this situation if you are using .NET / C# the best solution is just to use  Visual Studio Online REST API in codeplex .

Visual Studio Online REST API is a C# library for using TFS REST API in your .NET applications. Current version supports:

  • Projects and teams
  • Areas and iterations
  • Work item
  • Query and query folder
  • Version Control
  • Git
  • Build

This post is mainly focused on how to work with work items:

  • Create / update / read work items
  • Add / remove work item links and attachments
  • Create / update / delete tags

Before to start manage work items using Visual Studio Online REST API you need to do some preparations:

  • Prerequisites:

Authentication for the REST APIs:

These APIs support OAuth for authorization and you should plan to use that. With Oauth your users don’t have to provide their Visual Studio Online credentials to use when the APIs are called. To get started on your app, though, you can authenticate using basic authentication. You’ll have to enable alternate credentials to work with basic auth.

1. Open your profile

profile[1]

2. Enable alternate credentials on the credentials tab

enableCredentials[1]

3. Set your credentials

One appropriate place to do that are the application settings. It is possible of course to have a custom approach, but the samples from this blog assumes that you set there your credentials.

VSO-WorkItems-Pic01

 

VSO-WorkItems-Pic02

 Managing work items in Visual Studio Online

We will cover the main functionalities in the context of , regarding to the Visual Studio Online REST API

  • Get a team project description

One of the main features is to get the information for the VSO Team project, The code below shows how to do that,

   1: VsoClient _vsoClient;

   2: IVsoProject _client;

   3:  

   4: _vsoClient = new VsoClient(Properties.Settings.Default.AccountName, new NetworkCredential(Properties.Settings.Default.UserName, Properties.Settings.Default.Password));

   5: _client = _vsoClient.GetService<IVsoProject>();

   6:  

   7: var projects = _client.GetTeamProjects().Result;

   8: var project = _client.GetTeamProject(projects[0].Id.ToString(), true).Result;

   9: Console.Write("Project=" + project.ToString());

  10: Console.Write("\n");

  11: var descr = project.Description;

  12: Console.Write("Project Descr=" + descr);

 

The sample app is a console application. You can see the result screen below.

VSO-WorkItems-Pic03 

  • Getting the types of all work items

Another useful feature is how to get all   the types of the work items. It is possible for each item using the GetWorkItemType() method.

   1: _vsoClient = new VsoClient(Properties.Settings.Default.AccountName, new NetworkCredential(Properties.Settings.Default.UserName, Properties.Settings.Default.Password));

   2: _clientWit = _vsoClient.GetService<IVsoWit>();

   3:  

   4: var workItemTypes = _clientWit.GetWorkItemTypes(Properties.Settings.Default.ProjectName).Result;

   5: Console.Write("workItemTypes=" + workItemTypes.ToString());

   6: Console.Write("\n");

   7: for(int i = 0; i < workItemTypes.Count; i++)

   8: {

   9:     var workItemTypeEl = _clientWit.GetWorkItemType(Properties.Settings.Default.ProjectName, workItemTypes.Items[i].Name).Result;

  10:     Console.Write("workItemType["+i+"]=" + workItemTypeEl.ToString());

  11:     Console.Write("\n");

  12: }

  13: var workItemType = _clientWit.GetWorkItemType(Properties.Settings.Default.ProjectName, workItemTypes.Items[0].Name).Result;

  14:  Console.Write("workItemType[0]=" + workItemType.ToString());

  15: Console.Write("\n");

 

Below you can se the result in the console application:

VSO-WorkItems-Pic04

  • Get a field from a specified work item:
   1: VsoClient _vsoClient;

   2: IVsoWit _clientWit;

   3:  

   4: _vsoClient = new VsoClient(Properties.Settings.Default.AccountName, new NetworkCredential(Properties.Settings.Default.UserName, Properties.Settings.Default.Password));

   5: _clientWit = _vsoClient.GetService<IVsoWit>();

   6:  

   7: var fields = _clientWit.GetFields().Result;

   8: Console.Write("fields=" + fields.ToString());

   9: var field = _clientWit.GetField(fields.Items[0].ReferenceName).Result;

  10: Console.Write("field[0]=" + fields.ToString());

 

  • Get a specified backlog item:

It is possible to get a specified work item by work item id using GetWorkItem(workItemId) method

   1: VsoClient _vsoClient;

   2: IVsoWit _clientWit;

   3:  

   4: _vsoClient = new VsoClient(Properties.Settings.Default.AccountName, new NetworkCredential(Properties.Settings.Default.UserName, Properties.Settings.Default.Password));

   5: _clientWit = _vsoClient.GetService<IVsoWit>();

   6:  

   7: var workItem = _clientWit.GetWorkItem(170);

   8: Console.Write("workItem=" + workItem.ToString());

   9: Console.Write("\n");

  10: var witType = workItem.Result.Fields["System.WorkItemType"];

  11: Console.Write("WIT Type =" + witType.ToString());

  12: Console.Write("\n");

  13: var witAreaPath = workItem.Result.Fields["System.AreaPath"];

  14: Console.Write("WIT Area Path =" + witAreaPath.ToString());

  15: Console.Write("\n");

  16:  var witState = workItem.Result.Fields["System.State"];

  17:  Console.Write("WIT State =" + witState.ToString());

  18: Console.Write("\n");

  19:  var witTitle = workItem.Result.Fields["System.Title"];

  20:  Console.Write("WIT Title =" + witTitle.ToString());

  21: Console.Write("\n");

 

VSO-WorkItems-Pic05 

  • Create and execute queries in VSO / TFS Online using VSO REST API

Queries help you find work items that you want to review, triage, update, or generate a report.

Use the search box to find work items. Enter the ID or use filters. If you want a flat list of work items, a hierarchical list using a tree query, or a list showing dependencies using a direct links query, use the query editor to choose the query type.

You can create queries in Visual Studio Online, Team Web Access (TWA), and Team Explorer. Also, you can open a query in Excel or Project to perform bulk modifications.

   1: VsoClient _vsoClient;

   2: IVsoWit _clientWit;

   3:  

   4: _vsoClient = new VsoClient(Properties.Settings.Default.AccountName, new NetworkCredential(Properties.Settings.Default.UserName, Properties.Settings.Default.Password));

   5: _clientWit = _vsoClient.GetService<IVsoWit>()

   6:  

   7: const string LINK_QUERY = "Select System.Id, System.Title, System.State From WorkItems Where ([System.WorkItemType] = 'Product Backlog Item' AND [State] <> 'Closed' AND [State] <> 'Removed')";

   8: var ResultItems = _clientWit.RunFlatQuery(Properties.Settings.Default.ProjectName, LINK_QUERY).Result;

   9: for(int i =0; i < ResultItems.WorkItems.Count; i++)

  10: { 

  11:     var _id = ResultItems.WorkItems[i].Id;

  12:     Console.Write("\n");

  13:     Console.Write("WIT id =" + _id.ToString());

  14:     var _workItem = _clientWit.GetWorkItem(_id);

  15:     Console.Write("\n");

  16:     var _witAreaPath = _workItem.Result.Fields["System.AreaPath"];

  17:     Console.Write("WIT[" + _id  + "] Area Path =" + _witAreaPath.ToString());

  18:     Console.Write("\n");

  19:     var _witState = _workItem.Result.Fields["System.State"];

  20:     Console.Write("WIT[" + _id + "] State =" + _witState.ToString());

  21:     Console.Write("\n");

  22:     var _witTitle = _workItem.Result.Fields["System.Title"];

  23:     Console.Write("WIT[" + _id + "] Title =" + _witTitle.ToString());

  24:     Console.Write("\n");

  25: }

 

You can see the result in the test application console below:

VSO-WorkItems-Pic06

  • Create and update a work item

It is possible to use methods CreateWorkItem() and UpdateWorkItem() to create and update work items. The sample demo app demonstrates how to do that using the Visual Studio Online REST API. The initial state of the sample work items is shown below:

VSO-WorkItems-Pic07

Sample code also demonstrates how to create relations between work items:

   1: VsoClient _vsoClient;

   2: IVsoWit _clientWit;

   3:  

   4: _vsoClient = new VsoClient(Properties.Settings.Default.AccountName, new NetworkCredential(Properties.Settings.Default.UserName, Properties.Settings.Default.Password));

   5: _client = _vsoClient.GetService<IVsoWit>()

   6:  var orkItemId = 170;

   7: var workItems = _client.GetWorkItems(new int[] { workItemId }, RevisionExpandOptions.all).Result;

   8:  

   9: // Create new work item

  10: var bug = new WorkItem();

  11: bug.Fields["System.Title"] = "Sample bug N3";

  12: bug.Fields["System.History"] = DateTime.Now.ToString();

  13: bug = _client.CreateWorkItem(Properties.Settings.Default.ProjectName, "Bug", bug).Result;

  14:  

  15: var other = workItems[0];

  16:  

  17: // Update fields, add a link

  18: bug.Fields["System.Title"] = bug.Fields["System.Title"] + " (updated)";

  19: bug.Fields["System.Tags"] = "Demo Tag";

  20: Console.Write("\n");

  21: Console.Write("WIT id =" + workItemId.ToString());

  22: Console.Write("WIT[" + workItemId + "] Title =" + bug.Fields["System.Title"].ToString());

  23:  

  24: bug.Relations.Add(new WorkItemRelation()

  25: {

  26:     Url = other.Url,

  27:     Rel = "System.LinkTypes.Related",

  28:     Attributes = new RelationAttributes() { Comment = "Test Bug Relation" }

  29: });

  30:  

  31: bug = _client.UpdateWorkItem(bug).Result;

 

VSO-WorkItems-Pic08

The final result is shown below: a new bug: Sample Bug N3, updated and including a to another bug item.

VSO-WorkItems-Pic09

VSO-WorkItems-Pic09a 

  • Remove a work item ( bug)

It is also often seen case when you want to remove a work item. Actually it is not possible to remove it, but you can change the item state to “Removed”.

VSO-WorkItems-Pic10

   1: VsoClient _vsoClient;

   2: IVsoWit _client;

   3:  

   4: _vsoClient = new VsoClient(Properties.Settings.Default.AccountName, new NetworkCredential(Properties.Settings.Default.UserName, Properties.Settings.Default.Password));

   5: _client = _vsoClient.GetService<IVsoWit>()

   6:  

   7: const string LINK_QUERY = "Select System.Id, System.Title, System.State From WorkItems Where ([System.WorkItemType] = 'Bug')";

   8: var ResultItems3 = _client.RunFlatQuery(Properties.Settings.Default.ProjectName, LINK_QUERY).Result;

   9:  

  10: var workItemTitle = "Sample bug N2"

  11:  

  12: for (int i = 0; i < ResultItems3.WorkItems.Count; i++)

  13: {

  14:     var _id = ResultItems3.WorkItems[i].Id;

  15:     

  16:     Console.Write("\n");

  17:     Console.Write("WIT id =" + _id.ToString());

  18:     var _workItem = _client.GetWorkItem(_id).Result;

  19:     if (_workItem.Fields["System.Title"].ToString().Contains(workItemTitle))

  20:     {

  21:         Console.Write("\n");

  22:         var _witTitle = _workItem.Fields["System.Title"];

  23:         Console.Write("WIT[" + _id + "] Title =" + _witTitle.ToString());

  24:         Console.Write("\n");

  25:         var _witState = _workItem.Fields["System.State"];

  26:         

  27:         Console.Write("WIT[" + _id + "] State =" + _witState.ToString());

  28:         Console.Write("\n");

  29:         _workItem.Fields["System.State"] = "Removed";

  30:         var bug = _client.UpdateWorkItem(_workItem).Result;

  31:         Console.Write("\n");

  32:         _witState = _workItem.Fields["System.State"];

  33:         Console.Write("WIT[" + _id + "] Updated State =" + _witState.ToString());

  34:         Console.Write("\n");

  35:     }

  36:  

  37:  

  38: }

 

VSO-WorkItems-Pic11

The screen below shows the state after the execution of the last snippet: The item with the state, equal to “Removed” is not displayed in the list with work items:

VSO-WorkItems-Pic12

Below you can see the code of the whole application:

   1: using System;

   2: using System.Collections.Generic;

   3: using System.Linq;

   4: using System.Text;

   5: using System.Threading.Tasks;

   6: using System.Net;

   7: using VisualStudioOnline.Api.Rest.V1.Client;

   8: using VisualStudioOnline.Api.Rest.V1.Model;

   9:  

  10: namespace DemoApp

  11: {

  12:     class Program

  13:     {

  14:         static void Main(string[] args)

  15:         {

  16:             VsoClient _vsoClient;

  17:             IVsoProject _client;

  18:             IVsoWit _clientWit;

  19:  

  20:             _vsoClient = new VsoClient(Properties.Settings.Default.AccountName, new NetworkCredential(Properties.Settings.Default.UserName, Properties.Settings.Default.Password));

  21:             _client = _vsoClient.GetService<IVsoProject>();

  22:             _clientWit = _vsoClient.GetService<IVsoWit>();

  23:  

  24:  

  25:  

  26:             if (args != null)

  27:             {

  28:                 Console.WriteLine("args is null"); // Check for null array

  29:  

  30:                 var option = args[0];

  31:  

  32:                 switch (option)

  33:                 {

  34:                     case "firstproject":

  35:                         var projects = _client.GetTeamProjects().Result;

  36:                         var project = _client.GetTeamProject(projects[0].Id.ToString(), true).Result;

  37:                         Console.Write("Project=" + project.ToString());

  38:                         Console.Write("\n");

  39:                         var descr = project.Description;

  40:                         Console.Write("Project Descr=" + descr);

  41:                         break;

  42:  

  43:                     case "workitemtypes":

  44:                         var workItemTypes = _clientWit.GetWorkItemTypes(Properties.Settings.Default.ProjectName).Result;

  45:                         Console.Write("workItemTypes=" + workItemTypes.ToString());

  46:                         Console.Write("\n");

  47:                         for(int i = 0; i < workItemTypes.Count; i++)

  48:                         {

  49:                             var workItemTypeEl = _clientWit.GetWorkItemType(Properties.Settings.Default.ProjectName, workItemTypes.Items[i].Name).Result;

  50:                             Console.Write("workItemType["+i+"]=" + workItemTypeEl.ToString());

  51:                             Console.Write("\n");

  52:                         }

  53:                         var workItemType = _clientWit.GetWorkItemType(Properties.Settings.Default.ProjectName, workItemTypes.Items[0].Name).Result;

  54:                          Console.Write("workItemType[0]=" + workItemType.ToString());

  55:                         Console.Write("\n");

  56:  

  57:                         break;

  58:  

  59:                     case "getfields":

  60:                         var fields = _clientWit.GetFields().Result;

  61:                         Console.Write("fields=" + fields.ToString());

  62:                         var field = _clientWit.GetField(fields.Items[0].ReferenceName).Result;

  63:                         Console.Write("field[0]=" + fields.ToString());

  64:  

  65:                         break;

  66:                     case "getbacklogitem":

  67:                         var workItem = _clientWit.GetWorkItem(170);

  68:                         Console.Write("workItem=" + workItem.ToString());

  69:                         Console.Write("\n");

  70:                         var witType = workItem.Result.Fields["System.WorkItemType"];

  71:                         Console.Write("WIT Type =" + witType.ToString());

  72:                         Console.Write("\n");

  73:                         var witAreaPath = workItem.Result.Fields["System.AreaPath"];

  74:                         Console.Write("WIT Area Path =" + witAreaPath.ToString());

  75:                         Console.Write("\n");

  76:                          var witState = workItem.Result.Fields["System.State"];

  77:                          Console.Write("WIT State =" + witState.ToString());

  78:                         Console.Write("\n");

  79:                          var witTitle = workItem.Result.Fields["System.Title"];

  80:                          Console.Write("WIT Title =" + witTitle.ToString());

  81:                         Console.Write("\n");

  82:  

  83:                         break;

  84:  

  85:                     case "TestRunQuery":

  86:  

  87:                         const string LINK_QUERY2 = "Select System.Id, System.Title, System.State From WorkItems Where ([System.WorkItemType] = 'Product Backlog Item' AND [State] <> 'Closed' AND [State] <> 'Removed')";

  88:                         var ResultItems2 = _clientWit.RunFlatQuery(Properties.Settings.Default.ProjectName, LINK_QUERY2).Result;

  89:                         var _id2 = ResultItems2.WorkItems[0].Id;

  90:                         var _workItem2 = _clientWit.GetWorkItem(_id2);

  91:                         Console.Write("\n");

  92:                         var _witAreaPath2 = _workItem2.Result.Fields["System.AreaPath"];

  93:                         Console.Write("WIT Area Path =" + _witAreaPath2.ToString());

  94:                         Console.Write("\n");

  95:                         var _witState2 = _workItem2.Result.Fields["System.State"];

  96:                         Console.Write("WIT State =" + _witState2.ToString());

  97:                         Console.Write("\n");

  98:                         var _witTitle2 = _workItem2.Result.Fields["System.Title"];

  99:                         Console.Write("WIT Title =" + _witTitle2.ToString());

 100:                         Console.Write("\n");

 101:                         Console.Write("Flat Query Result =" + ResultItems2.ToString());

 102:                         Console.Write("\n");                        

 103:  

 104:                         break;

 105:                     case "TestRunQueries":

 106:  

 107:                         const string LINK_QUERY = "Select System.Id, System.Title, System.State From WorkItems Where ([System.WorkItemType] = 'Product Backlog Item' AND [State] <> 'Closed' AND [State] <> 'Removed')";

 108:                         var ResultItems = _clientWit.RunFlatQuery(Properties.Settings.Default.ProjectName, LINK_QUERY).Result;

 109:                         for(int i =0; i < ResultItems.WorkItems.Count; i++)

 110:                         { 

 111:                             var _id = ResultItems.WorkItems[i].Id;

 112:                             Console.Write("\n");

 113:                             Console.Write("WIT id =" + _id.ToString());

 114:                             var _workItem = _clientWit.GetWorkItem(_id);

 115:                             Console.Write("\n");

 116:                             var _witAreaPath = _workItem.Result.Fields["System.AreaPath"];

 117:                             Console.Write("WIT[" + _id  + "] Area Path =" + _witAreaPath.ToString());

 118:                             Console.Write("\n");

 119:                             var _witState = _workItem.Result.Fields["System.State"];

 120:                             Console.Write("WIT[" + _id + "] State =" + _witState.ToString());

 121:                             Console.Write("\n");

 122:                             var _witTitle = _workItem.Result.Fields["System.Title"];

 123:                             Console.Write("WIT[" + _id + "] Title =" + _witTitle.ToString());

 124:                             Console.Write("\n");

 125:                         }

 126:  

 127:                         break;

 128:  

 129:                     case "CreateAndUpdateWorkItem":

 130:  

 131:  

 132:  

 133:                         CreateAndUpdateWorkItem(_clientWit, 170);

 134:                         break;

 135:  

 136:                     case "RemoveBugItem":

 137:  

 138:                         RemoveBugItem(_clientWit, "Sample Bug N2");

 139:  

 140:                         break;

 141:  

 142:                 }

 143:             }

 144:             

 145:  

 146:             Console.ReadLine();

 147:  

 148:         }

 149:  

 150:         public static void CreateAndUpdateWorkItem(IVsoWit _client, int workItemId)

 151:         {

 152:             var workItems = _client.GetWorkItems(new int[] { workItemId }, RevisionExpandOptions.all).Result;

 153:  

 154:             // Create new work item

 155:             var bug = new WorkItem();

 156:             bug.Fields["System.Title"] = "Sample bug N3";

 157:             bug.Fields["System.History"] = DateTime.Now.ToString();

 158:             bug = _client.CreateWorkItem(Properties.Settings.Default.ProjectName, "Bug", bug).Result;

 159:  

 160:             var other = workItems[0];

 161:  

 162:             // Update fields, add a link

 163:             bug.Fields["System.Title"] = bug.Fields["System.Title"] + " (updated)";

 164:             bug.Fields["System.Tags"] = "Demo Tag";

 165:             Console.Write("\n");

 166:             Console.Write("WIT id =" + workItemId.ToString());

 167:             Console.Write("WIT[" + workItemId + "] Title =" + bug.Fields["System.Title"].ToString());

 168:  

 169:             bug.Relations.Add(new WorkItemRelation()

 170:             {

 171:                 Url = other.Url,

 172:                 Rel = "System.LinkTypes.Related",

 173:                 Attributes = new RelationAttributes() { Comment = "Test Bug Relation" }

 174:             });

 175:  

 176:             bug = _client.UpdateWorkItem(bug).Result;

 177:         }

 178:  

 179:  

 180:         public static void RemoveBugItem(IVsoWit _client, int workItemId)

 181:         {

 182:             //"Sample bug N2 (updated)"

 183:             const string LINK_QUERY = "Select System.Id, System.Title, System.State From WorkItems Where ([System.WorkItemType] = 'Bug' AND [State] <> 'Closed' AND [State] <> 'Removed')";

 184:             var ResultItems = _client.RunFlatQuery(Properties.Settings.Default.ProjectName, LINK_QUERY).Result;

 185:             var workItems = _client.GetWorkItems(new int[] { workItemId }, RevisionExpandOptions.all).Result;

 186:             // Create new work item

 187:             var bug = new WorkItem();

 188:             bug.Fields["System.Title"] = "Sample bug N3";

 189:             bug.Fields["System.History"] = DateTime.Now.ToString();

 190:             bug = _client.CreateWorkItem(Properties.Settings.Default.ProjectName, "Bug", bug).Result;

 191:  

 192:             var other = workItems[0];

 193:  

 194:             // Update fields, add a link

 195:             bug.Fields["System.Title"] = bug.Fields["System.Title"] + " (updated)";

 196:             bug.Fields["System.Tags"] = "Demo Tag";

 197:             Console.Write("\n");

 198:             Console.Write("WIT id =" + workItemId.ToString());

 199:             Console.Write("WIT[" + workItemId + "] Title =" + bug.Fields["System.Title"].ToString());

 200:  

 201:             bug.Relations.Add(new WorkItemRelation()

 202:             {

 203:                 Url = other.Url,

 204:                 //Rel = "System.LinkTypes.Dependency-Forward",

 205:                 Rel = "System.LinkTypes.Related",

 206:                 Attributes = new RelationAttributes() { Comment = "Test Буг Relation" }

 207:             });

 208:  

 209:             bug = _client.UpdateWorkItem(bug).Result;

 210:         }

 211:  

 212:         public static void RemoveBugItem(IVsoWit _client, string workItemTitle)

 213:         {

 214:             //"Sample bug N2 (updated)"

 215:             const string LINK_QUERY = "Select System.Id, System.Title, System.State From WorkItems Where ([System.WorkItemType] = 'Bug')";

 216:             var ResultItems3 = _client.RunFlatQuery(Properties.Settings.Default.ProjectName, LINK_QUERY).Result;

 217:  

 218:             for (int i = 0; i < ResultItems3.WorkItems.Count; i++)

 219:             {

 220:                 var _id = ResultItems3.WorkItems[i].Id;

 221:                 

 222:                 Console.Write("\n");

 223:                 Console.Write("WIT id =" + _id.ToString());

 224:                 var _workItem = _client.GetWorkItem(_id).Result;

 225:                 if (_workItem.Fields["System.Title"].ToString().Contains(workItemTitle))

 226:                 {

 227:                     Console.Write("\n");

 228:                     var _witTitle = _workItem.Fields["System.Title"];

 229:                     Console.Write("WIT[" + _id + "] Title =" + _witTitle.ToString());

 230:                     Console.Write("\n");

 231:                     var _witState = _workItem.Fields["System.State"];

 232:                     

 233:                     Console.Write("WIT[" + _id + "] State =" + _witState.ToString());

 234:                     Console.Write("\n");

 235:                     _workItem.Fields["System.State"] = "Removed";

 236:                     var bug = _client.UpdateWorkItem(_workItem).Result;

 237:                     Console.Write("\n");

 238:                     _witState = _workItem.Fields["System.State"];

 239:                     Console.Write("WIT[" + _id + "] Updated State =" + _witState.ToString());

 240:                     Console.Write("\n");

 241:                 }

 242:  

 243:  

 244:             }

 245:         }

 246:     }

 247: }

 

If you want more information about haw to manage programmatically  Visual Studio Online work otems feel free to contact me at michael@mateev.net Follow my blog : mmateev.infragistics.com  .

You can learn more about the PASS events if you follow me on Twitter @mihailmateev  , and stay in touch on Facebook, Google+, LinkedIn and Bulgarian BI and .Net User Group !

mvp

Posted in .Net, Azure, C#, Cloud, Kanban, Microsoft Azure, Scrum, TFS, TFS Online, Visual Studio 2013, Visual Studio Online, VSO | Leave a comment

CEE MVP Open Days 2015

The event was held on Sunday and Monday: 29 – 30 of March of March 2015 in Belgrade . Serbia. Each year the Microsoft MVPs from each region meet at special “MVP Open Days” event, meeting also professionals from Microsoft Product teams. This was f the biggest Microsoft MVP Open Days for the Central Eastern Europe Region. MVPs met also specialist from the Microsoft Development Lab in Serbia.

Many special thanks to Alessandro Teglia ( EMEA Regional Manager for the MVP Award Program @Microsoft )  , Yulia Belyanina ( Community Program Manager, CEE @Microsoft) and Marina Terzi ( MVP coordinator @ Microsoft ) who did a lot this event to be organized.

This was my first event as a speakers after I joined Strypes / ICT Automatisering . I’m glad that my sessions about modern IoT solutions with Microsoft Azure was included in the schedule. That gave me opportunity to share some impressions about solutions architecture and best practices regarding to the experience gain in Strypes and ICT Automasering .

You can enjoy some photos from the event below:

11096571_10206343973868919_8300165194104039360_n[1]

11081368_10206345080576586_39145765400419315_n[1]

11103021_781087831959602_7602724336484114163_n[1]

11082555_10206344808929795_4758352264124318729_n[1]

11081073_781087955292923_2363353690240429989_n[1]

10665200_10204931179692789_3024002307507526831_n[1]

Event Recap:

– It was the MVP Open Days event with most ever MVPs – participants from CEE

– There was 3 sessions from MVPs from Bulgaria

– My session on “Think Connected – Modern IoT Solutions with Microsoft Azure” was included in the track with technical presentations

You can learn more about the community events in CEE if you follow me on Twitter @mihailmateev , and stay in touch on Facebook, Google+, LinkedIn and Bulgarian BI and .Net User Group !

pass_logo_partner_bw[1] Strypes-LOGO+payoff_FC

Posted in .Net, Azure, Azure NoSQL, Azure Storage, ceeod, COD15, ICT, ict.eu, Internet of Things, IoT, Microsoft, Microsoft Azure, Microsoft cloud, MVP, MVPBuzz, Strypes, Strypes.eu | Leave a comment