VMware Cloud on AWS: What’s New in January 2022 – Cloud Blog – VMware


First of all, Happy New Year everyone. Hope you all had a nice break with your families over the holidays.

Now, let’s talk more about VMware Cloud on AWS. VMware Cloud on AWS is a jointly engineered hybrid cloud service that extends on-premises vSphere environments to a VMware Software-Defined Data Center (SDDC) running on Amazon Elastic Compute Cloud (Amazon EC2) elastic, bare-metal infrastructure that is fully integrated as part of AWS. Customers can spin up a VMware Cloud on AWS SDDC in 20 Global AWS Regions (including a special controlled instance running in AWS GovCloud (US-East) and AWS GovCloud (US-West)) in typically under 2 hours and scale capacity within minutes on average. With its consistent infrastructure and operations, efficient resource utilization and on-demand consumption options, it has been helping customers save on total cost of ownership compared to traditional, distributed on-premises infrastructure. You can check out this recent regional bank spotlight by Forrester to see how VMware Cloud on AWS helped this regional bank in saving hundreds of thousands of dollars on power, cooling, and upkeep, which accounted for 10% of the organization’s technology budget.

Since the launch of the service 4 years back, every quarter, we have been delivering lot of new features and capabilities based on customer requirements. Now, let’s dive into what’s new in VMware Cloud on AWS in VMware’s Fiscal Year 2022’Q4

Enhancements to enterprise workload support:

  • PCI DSS Certification for VMware Cloud on AWS – Expanding regional availability in 5 more AWS regions: VMware Cloud on AWS received the highest level of PCI certification (PCI DSS Level 1 provider status). In phase 4, PCI compliant SDDCs will be enabled in 5 more AWS regions: AWS South America (Sao Paulo), AWS Europe (Paris), AWS Europe (Stockholm), AWS Asia Pacific (Mumbai) and AWS Asia Pacific (Seoul) that makes PCI compliant SDDC availability to 18 regions in total- AWS US West (Oregon), AWS US West (N. California), AWS US East (N. Virginia), AWS US East (Ohio), AWS Canada (Central), AWS South America (Sao Paulo), AWS Europe (Paris), AWS Europe (Frankfurt), AWS Europe (Milan), AWS Europe (Ireland), AWS Europe (London), AWS Europe (Stockholm), AWS Asia Pacific (Sydney), AWS Asia Pacific (Singapore), AWS Asia Pacific (Tokyo), AWS Asia Pacific (Osaka), AWS Asia Pacific (Mumbai) and AWS Asia Pacific (Seoul). By being certified as PCI DSS compliant level 1 service provider, VMware Cloud on AWS service operates in compliance with PCI DSS compliant security measures and controls, thereby potentially addressing the needs of a broad range of customers and workloads that need to store, process, or transmit cardholder or sensitive authentication data. VMware Cloud on AWS PCI DSS compliant infrastructure enables customers to evacuate PCI in-scope systems from their data centers with deployment, configuration and management of PCI enabled SDDCs. The VMware Cloud on AWS PCI DSS compliant solution reduces the time, effort, cost and complexity associated with operating applications requiring PCI.

Please note that this PCI DSS compliance certification is not applicable to any VMware Cloud on AWS add-ons. It is only applicable to core SDDC components (vCenter, ESXi, vSAN and NSX).

  • OSPAR(Singapore) compliance: The Association of Banks (ABS) in Singapore have created a set of guidelines and control procedures for outsourced service providers providing services to financial institutions in Singapore. The Outsourced Service Provider Auditor Report (OSPAR) demonstrates how we address the requirements in these ABS guidelines. VMware Cloud on AWS has completed an external audit and obtained the OSPAR attestation. OSPAR attestation gives our customers and their auditors assurance on how we manage and support security, operations and compliance at VMware in line with ABS guidelines
  • VMware Telco Cloud Platform™ – Public Cloud: Powered by VMware Cloud on AWS, VMware Telco Cloud Platform – Public Cloud enables Service Providers (SPs) to migrate both telco and IT workloads to VMware Cloud on AWS quickly, easily, and cost efficiently, providing SPs flexibility of the workload placement to maximize their business growth of today and the future. 

With VMware Telco Cloud Platform – Public Cloud, the infrastructure expansion with VMware Cloud on AWS is on-demand, providing the agility and speed needed for the 5G deployments on a massive scale. This results in faster time to market for the new 5G services as well as the new market entry without unnecessary costs and risks. Check out this latest Reference Architecture to learn more from Architecture perspective

Enhancements in networking, storage, availability, and resiliency:

  • VMware Transit Connect Intra-Region Peering with AWS Transit Gateway: This feature enables VMware Cloud on AWS SDDC customers to peer their VMware Transit Connect with AWS Transit Gateway (TGW) in the same region. With this, customers can establish high bandwidth connectivity between their VMC SDDC Group and AWS TGW, enabling seamless access to AWS VPC resources at scale without the need for a Transit VPC. This further simplifies access between VMware Cloud on AWS resources and AWS VPC resources, while retaining control over connectivity in the respective environments. Learn more
  • Amazon FSx with NetApp ONTAP Integration: Announcing preview of Amazon FSx with NetApp ONTAP Integration – an NFS datastore for VMware Cloud on AWS. With this capability, you can attach a fully managed, low cost, high performance NFS datastore built on NetApp’s ONTAP file system to the VMware Cloud on AWS SDDC and scale their storage environment as needed without the need to purchase additional host instances. Amazon FSx for NetApp ONTAP offers high-performance file storage with compression and deduplication to help you further reduce storage costs. It provides ONTAP’s data management capabilities, like snapshots, clones, and replication across your hybrid cloud environment that will improve staff productivity and responsiveness. Learn more. We are opening early access nominations for this capability now. So, if you are interested in learning more about the early access program, please send an email to vmc_fsx_ea@vmware.com
  • Elastic DRS baseline policy: Elastic DRS monitors the current demand on VMware Cloud on AWS SDDC and maintains an optimal number of provisioned hosts to keep cluster utilization high while maintaining desired CPU, memory, and storage performance. Now, with the introduction of Elastic DRS baseline policy, it will add hosts to an existing SDDC if less than 20% free capacity is available on any vSAN cluster or if the Availability Zone failure occurs.

Please note: Elastic DRS baseline policy is always running and cannot be disabled. But you can add more rules by selecting an additional Elastic DRS policy.

  • 3+ hosts SDDC Cluster scale down: Customers can start their hybrid cloud journey by starting small with 2-host i3.metal or i3en.metal clusters as the primary clusters within an SDDC as well as secondary clusters within an existing SDDC and they can scale up as per their needs. However, scaling down from 3+ hosts to 2-hosts SDDC cluster was not supported previously. With the introduction of this new capability, customers will be able to scale down 3+ hosts SDDC cluster to 2-hosts in single AZ primary or secondary clusters. Once scale down is completed, EDRS Baseline policy will be enabled. This feature will be useful in scenarios such as:
    • Customer scales up the cluster by adding additional hosts for seasonal spikes in demand or a temporary event and wants to scale it down when season or event is over
    • Elastic DRS scales out a 2-node cluster automatically for vSAN slack space requirement and customer wants to scale it back to the 2-hosts cluster when vSAN Slack space is above 20%

Please note: If host removal triggers the EDRS scale-up event, then customers won’t be able to scale down the cluster to 2-hosts. Scaling down a a cluster with 8 CPU cores is not allowed.

  • Elastic DRS Storage Scale-up threshold update: With this enhancement, the vSAN Slack Space requirement has been decreased from 30% to 20%. To accommodate this improvement, the Storage Scale-up threshold for all Elastic DRS policies has been increased to 80%. Customers can now consume up to 79% of vSAN capacity regardless of the Elastic DRS policy.
  • Enhancements to VMware Cloud Disaster Recovery: As a recap, VMware Cloud Disaster Recovery offers on-demand disaster recovery to IT administrators responsible for IT infrastructure and services resiliency, delivered as an easy-to-use SaaS solution with cloud economics. It combines cost-efficient cloud storage with simple SaaS-based management for IT resiliency at scale, through simple testing and orchestration of failover and failback plans. Customers benefit from consistent VMware operations across production and DR sites and a ‘pay when you need’ failover capacity model for DR resources, while benefiting from instant power-on capabilities for fast recovery after disaster events, including ransomware attacks. Some of the key capabilities include pilot light feature for faster recovery times with minimal cloud footprint, Instant Power-On for VMs in the cloud in the event of a failover, optimized delta-based failback—all within a familiar VMware environment, reducing the need for IT re-training and driving ease of use as a core component of resiliency. SaaS-based management eliminates the need to deploy and lifecycle manage the DR software. Continuous DR health checks and built-in audit reports deliver a simplified operational experience and confidence in recovery readiness.
    • Low cost deployment options:
      • VMware Cloud DR now supports 2-host i3en.metal clusters for Pilot Light deployments for protecting storage intensive workloads, thus helping customers start small and optimize their failover infrastructure costs.
      • VMware Cloud DR now supports 2-host multi-clusters- a capability that provides deployment flexibility for customers looking to minimize their cloud failover footprint by keeping 2-host clusters versus 3-host clusters. It is also particularly useful for those looking to scale up the DR footprint for a test or failover commit and still preserve a 2-host Pilot light deployment after failing back, thus providing customers additional flexibility to customize their individual deployments to match their DR infrastructure needs.
    • Protect to multiple regions in a single org: VMware Cloud DR now supports deployments in multiple regions per org. This facilitates simplified billing and operations as everything can be managed from a single, federated, global DR console. Each orchestrator can support up to four cloud filesystems, each of which could be mapped onto a recovery SDDC (in the same AZ). For more details on air-gapped Scale Out Cloud Filesystem (SCFS), refer to this blog.
    • ISO 27001/27017/27018 compliance: VMware Cloud DR has achieved the International Organization for Standardization (ISO) certificate, supporting 27001, 27017, and 27018 standards. ISO is an independent, non-governmental international organization who brings together experts to share knowledge and develop international standards that support innovation and provide solutions to global challenges. Please visit the VMware Cloud Trust Center to learn more and to download the ISO certificate.
    • Cyber Essentials Plus compliance: VMware Cloud DR has achieved the Cyber Essentials Plus certificate. Cyber Essentials is a UK Government-backed framework that helps protect organizations from different cyber-attacks. The Cyber Essentials Plus certification requires an accredited third part to conduct external vulnerability testing to ensure security systems are protected. Please visit the VMware Cloud Trust Center to learn more and to download the Cyber Essentials Plus certificate.
    • CSA Compliance: VMware Cloud DR is now a Cloud Security Alliance (CSA) Trusted Cloud Provider and registered as a Security, Trust, Assurance and Risk (STAR) Level One service. CSA is an organization dedicated to defining best practices to help ensure a secure cloud computing environment. STAR Registry is a publicly accessibly registry that documents the security and privacy controls. You can find the VMware Cloud DR STAR Register listing, including the downloadable CAIQ, here.
    • Interoperability with VMware HCX: With this capability, customers can simultaneously migrate and protect their on-premises workloads using VMware HCX and VMware Cloud DR —with no DR protection downtime.
    • UX enhancements: VMware Cloud DR has done multiple UX enhancements such as added operational visibility with improved replication progress reporting, connectivity checks between the source site and target and an In-Product Feedback feature.
  • Enhancements to VMware Site Recovery: VMware Site Recovery™ for VMware Cloud on AWS simplifies traditional disaster recovery and delivers a high-performance service. The service automates workload recovery in a DR event between on-premises data centers and VMware Cloud on AWS, as well as between different instances of VMware Cloud on AWS. Built on top of enterprise-grade DR tools (VMware Site Recovery Manager, vSphere Replication) and global cloud infrastructure (AWS), the service provides an end-to-end disaster recovery solution that is quick to deploy and leverages existing know-how. New enhancements include:
    • Support 3000 VMs per SDDC: Previously, customers were able to protect up to 1500 VMs per SDDC using VMware Site Recovery. With this enhancement, now they can simplify large DR deployments further and reduce their DR costs with 2X increase in VMs i.e., They can protect up to 3000 VMs per SDDC with VMware Site Recovery.
    • PCI DSS certification for VMware Site Recovery: VMware Site Recovery received the highest level of PCI certification (PCI DSS Level 1 provider status). By being certified as PCI DSS compliant level 1 service provider, VMware Site Recovery service operates in compliance with PCI DSS compliant security measures and controls, thereby potentially addressing the needs of a broad range of customers and workloads that need to store, process, or transmit cardholder or sensitive authentication data.

PCI compliance will be enabled in the AWS regions that support VMware Cloud on AWS where SDDCs are configured for compliance hardening for PCI.  See link  https://docs.vmware.com/en/VMware-Cloud-on-AWS/services/com.vmware.vmc-aws-operations/GUID-9708C514-30FE-4D75-A3E4-E358166EEB1F.html

Enhancements to commerce experience: 

VMware and AWS are constantly looking at ways to improve the experience of our joint customers. One key aspect of this is the commerce experience for customers that are purchasing the VMware Cloud on AWS service and associated value-added VMware solutions through AWS or its partner network. New enhancements planned in this area include:

  • Order Form Integration and automation through AWS Console: Today, customers who purchase VMware Cloud on AWS service from AWS have to go through a time-consuming manual ordering and onboarding process.

With this new capability, customers will be able to submit an order form through VMware Cloud on AWS tile available on AWS management console thus streamlining the onboarding process. This enhancement helps AWS customers get onboarded onto the service in fast and easy manner and gives them a single unified experience across VMware Cloud console and AWS management console.

Enhancements in migration capabilities:

  • VMware HCX: High availability of Network Extension: VMware HCX Network Extension bridges networks between VMware HCX and activated data centers by extending the Virtual Machine networks to a VMware HCX remote site. Previously, Network Extension service cannot survive unexpected appliance (VM) level failures such as failures due to kernel issues or underlying ESXi host issues and, there was no fault tolerant mechanism in place to automatically recover from such problems. With this new capability, VMware HCX now uses a High Availability (HA) setup with a pair of appliances both at the source and the destination site in a standby/active mode such that if the active appliance fails, traffic is rerouted to the standby appliance which becomes the new active appliance. This failover would be automatic in nature once the HA is configured and would ensure overall higher resiliency of the system., thus preventing any downtime in case of appliance failure.

Please note: This capability defends up to one failure event only. If more than one appliance fails in the same HA setup at the same time, the service will be disrupted. This capability is supported in VMware HCX 4.3.0 and above, but it will not be automatically enabled even after upgrading to HCX 4.3.0. For HA setup to work, the service mesh must contain enough fresh NE appliances and HA setup will not deploy any new NE appliances if there are not sufficient fresh NE appliances available in a service mesh.

Enhancements in subscription options:

  • Flexible subscription option when purchasing from AWS:

Customers can purchase VMware Cloud on AWS hosts from AWS as an on-demand or 1-year or 3-year subscription with the option to pay monthly or upfront. Previously, these subscription purchasing options were tied to a specific region and instance type for the entire duration – without the ability to change or modify these dimensions. With the introduction of the new flexible subscription offering for customers who purchase the service directly from VMware or through AWS, we provide greater flexibility to exchange their subscription as their business needs change.

Here are the feature details:

  1. By purchasing a flexible subscription, customers are entitled to terminate their existing flexible term subscription (1-year or 3-year commitment) early and utilize the value remaining for a purchase of a new 1-year or 3-year subscription. Please note: To avail this flexibility, customers must purchase new flexible 1-year or 3-year term commitments, paid upfront. They cannot change previously purchased non-flexible 1-year or 3-year term commitments.
  2. Some of the examples of flexible subscription exchange use cases are: 
    • Instance type: Customers can purchase a new term commitment for new instance type. This is specifically useful if customers would like to use existing subscription funds for a different use case where different instance type is more optimal. 
    • AWS Region: Customers can purchase a new term commitment for a different AWS region. This is specifically useful for the customers if they would like to utilize existing subscription funds for scaling IT infrastructure in new region as per their business needs such as footprint expansion in new regions, regional business growth etc.

Please note: The flexible subscription option is available for 1-year or 3-year term commitments only and not for on-demand option.

Enhancements to advanced cloud management:

  • CloudHealth support for VMware Cloud on AWS: CloudHealth provides a single platform with visibility into cost, usage, and performance of hybrid cloud and public cloud resources, including VMware vSphere, VMware Cloud on AWS, and AWS. With CloudHealth, organizations can simplify cloud financial management, streamline operations, and improve cross-organizational collaboration across their cloud environment. 

Key capabilities of this release include: 

  1. A summarized information on cost and inventory, with a snapshot of VMware Cloud on AWS historical cost for 13 months that helps users to understand trends in their spend. 
  2. A unified dashboard across clouds that serves as a single point of reference across VMware Cloud on AWS, vSphere, and public clouds. 
  3. Current and previous month billing reports that provide visibility into spend during the billing period, which might differ from calendar month period. 
  4. Compute, Elastic IP, and Data Transfer Usage history reports that give the ability to slice-and-dice service usage across CSP organizations, VMware Cloud on AWS Regions and services. 
  5. Visibility into VMware Cloud on AWS stack and comprehensive list of assets (from SDDC to virtual machines). 

vRealize Cloud Management

VMware vRealize® Cloud Management is an intelligent hybrid cloud management solution that enables consistent deployments and operations for apps, infrastructure, and platform services, across VMware Cloud on AWS, on-premises and/or hybrid cloud environments. The solution helps organizations accelerate application migration to VMware Cloud on AWS as well as innovate with quick and easy access to services, gain efficiency by improving visibility and automation, and improve control while mitigating risk through unified operations and governance.

VMware vRealize® Cloud Universal™ is a cloud management suite that combines on-premises and SaaS management into one license to help accelerate our customers’ business transition to the cloud. It gives customers the flexibility to deploy on-premises or SaaS, interchangeably, without the need to repurchase, for a consistent hybrid cloud management experience. In vRealize Cloud Universal, customers get vRealize Automation Cloud, vRealize Operations Cloud, vRealize Log Insight Cloud, vRealize Network Insight Cloud, and additional features exclusive to vRealize Cloud Universal. For more information, visit: https://www.vmware.com/products/vrealize-cloud-universal.html. Below are the enhancements for vRealize Cloud Management portfolio:

  • VMware vRealize Automation Cloud provides self-service IaaS consumption with governance for VMware Cloud on AWS. With vRealize Automation Cloud, customers can reduce the complexity of their IT environment, streamline IT processes and deliver a DevOps-ready automation platform. It enables automated workload provisioning by setting up a self-service infrastructure for developers and managing it with governance policies for better insight and control. It also delivers Infrastructure as Code-based automation for provisioning and management of SDDCs on VMware Cloud on AWS. VMware Cloud Templates (VCTs) created in vRealize Automation Cloud are written declaratively in YAML. Customers can automate SDDCs, virtual machines, networking, and other infrastructure components through VCTs, serverless functions (ABX) and other integrations. With GitLab and GitHub integration, blueprints and other automation scripts can be versioned and stored in a source code repository. New enhancements include:
    • Expanded regional availability: New geographic availability in Asia Pacific (Tokyo). This will be in addition to the available geographic locations for vRealize Automation Cloud, namely US West (Oregon), Europe (Frankfurt), Canada (Central), Europe (London) and South America (Sao Paulo), Asia Pacific (Singapore) and Asia Pacific (Sydney)
  • VMware vRealize Operations Cloud delivers self-driving IT operations management for private, hybrid cloud environments in a unified, AI-powered platform. Offering full-stack visibility from physical, virtual and cloud infrastructure – including VMs and containers – to the applications they support, vRealize Operations Cloud provides continuous performance optimization, efficient capacity and cost planning and management, app-aware intelligent remediation, and integrated compliance.
    • Expanded regional availability: New geographic availability in Asia Pacific (Tokyo). This will be in addition to the available geographic locations for vRealize Operations Cloud, namely US West (Oregon), Europe (Frankfurt), Canada (Central), Europe (London) and South America (Sao Paulo), Asia Pacific (Singapore) and Asia Pacific (Sydney)
  • vRealize Network Insight Cloud enhancements for VMware Cloud on AWS customers: vRealize Network Insight Cloud is a solution for end-to-end network visibility, troubleshooting, and predictive analytics that enables application migrations, optimizes network performance with troubleshooting capabilities, and manages the scaling of VMware Cloud on AWS deployments.
    • NSX Advanced Load Balancer (AVI) integration: vRealize Network Insight Cloud will have visibility and integration with the NSX Advanced Load Balancer when deployed in VMware Cloud on AWS as well as in on-premises deployments. This integration will provide richer application layer information. 
    • Guided Network Troubleshooting: Guided Network Troubleshooting is a new vRealize Network Insight capability that provides a streamlined experience focused primarily on the application as a start point for troubleshooting. This new feature will help automate troubleshooting of VMware Cloud on AWS dependencies for quicker network issue root cause. 
    • FIPS Compliance: FIPS based collection of FIPS approved ciphers will be supported by vRealize Network Insight Cloud. The Federal Information Processing Standard (FIPS) 140-2 is a U.S. and Canadian government standard that specifies security requirements for cryptographic modules. VMware has validated various cryptographic modules for the FIPS 140-2 standard. 
    • Streaming Databus support: Support for high speed Streaming Databus for sending the rich data and metrics from vRealize Network Insight Cloud externally to other receivers and destinations. 
  • VMware vRealize Log Insight Cloud enhancements: vRealize Log Insight Cloud delivers centralized log management, deep operational visibility, and intelligent analytics across your VMware SDDC software stack running in VMware Cloud on AWS.  With accelerated IT troubleshooting and better security across your private and VMC infrastructure, every subscription includes certain vRealize Log Insight Cloud features focused on audit and diagnostic capabilities with ability to upgrade to the full product for intuitive, actionable dashboards, sophisticated analytics and broad third-party extensibility. vRealize Log Insight Cloud is the one platform capable of bringing log data from your entire environment together—no matter where it resides—and extracting meaning from it. The solution brings order to the chaos of millions of unstructured data points, turning raw log data into actionable insights that can help you address both security and operational issues. vRealize Log Insight Cloud contains built-in knowledge and native support for VMware SDDC technologies—from VMware vSphere, NSX-T firewall, VMware Cloud on AWS logs to 3rd party technologies as Docker, MS SQL, Apache and many more. With more than 100 Content Packs for a broad range of VMware, VMware Cloud, Multi-Cloud, and 3rd party hardware and software connections and integrations, it is indisputably the best solution for your VMware environment.
    • Expanded regional availability: New geographic availability in Asia Pacific (Tokyo). This will be in addition to the available geographic locations for vRealize Log Insight Cloud, namely US West (Oregon), Europe (Frankfurt), Canada (Central), Europe (London) and South America (Sao Paulo), Asia Pacific (Singapore) and Asia Pacific (Sydney)
    • AWS Lamba and HashiCorp Vault Integration: Integration with AWS Lambda functions now enables you to forward logs from AWS CloudWatch, CloudTrail, and many other services to vRealize Log Insight Cloud. If you’re seeking more security with the API token and need to avoid storing your VMware vRealize Log Insight Cloud credentials in the AWS Lambda functions, you can now use the HashiCorp Vault integration for secrets management.

And with that, we have come to an end of the blog. Below you will find the status of the features listed above.

Availability

To view the latest status of features and release updates for VMware Cloud on AWS, visit: https://www.vmware.com/products/vmc-on-aws.html. And refer to the release notes VMware Cloud on AWS release notes.

The following capabilities are available today: OSPAR(Singapore) compliance, VMware Telco Cloud Platform™ – Public Cloud, VMware Transit Connect Intra-Region Peering with AWS Transit Gateway, Elastic DRS baseline policy, 3+ hosts SDDC Cluster scale down, Elastic DRS Storage Scale-up threshold update, VCDR: Low-cost deployment options, Protect to multiple regions in a single org, ISO 27001/27017/27018 compliance, Cyber Essentials Plus compliance, CSA Compliance, Interoperability with VMware HCX, UX enhancements, VSR: Support 3000 VMs per SDDC, PCI DSS certification for VSR, VMware HCX: High availability of Network Extension, Flexible subscription option when purchasing from AWS, vRealize Automation Cloud: AWS Asia Pacific(Tokyo) region, vRealize Operations Cloud: AWS Asia Pacific(Tokyo) region, vRealize Log Insight Cloud: AWS Asia Pacific(Tokyo) region, AWS Lamba and HashiCorp Vault Integration, vRealize Network Insight Cloud: NSX Advanced Load Balancer (AVI) integration, Guided Network Troubleshooting, FIPS Compliance, Streaming Databus support

The following capabilities/offerings are in Preview today: Amazon FSx with NetApp ONTAP Integration, CloudHealth support for VMware Cloud on AWS

The following capabilities/offerings are expected to be available in VMware’s Q4’Fiscal Year 2022: PCI DSS Certification: Expanding regional availability in 5 more AWS regions, Order Form Integration and automation through AWS Console

Resources

For more information related to VMware Cloud on AWS, here are some more learning resources for you:



Source link