zend test - Page 816 - Developer IT

Private subnet for VM server host-only network

- by Derek Pressnall

At my current job, we distribute a product based on a Linux server with multiple VMs defined (using KVM / libvirt). We are planning to expose limited ports to the customer's network, and use iptables to direct inbound traffic to the appropriate internal VM. My question: is there a class of private subnets that I can use for the internal host-only network that is least likely to conflict with a client IP subnet? Specifically, if I choose a /24 out of any of the RFC-1918 defined private subnets (such as 192.168.x.x), there is a chance of conflicting with a customer-used range. I noticed that several current VM implementations default to 192.168.122.x -- is this due to an RFC that I'm not familiar with, and therefore this is a safe range to use (that most network admins would avoid)? Or did the various VM vendors just pick that range randomly? I guess I'm looking for an IP range that is more private than the existing private (RFC1918) addresses. The only other thought I had was to use one of the "Test Net" IP ranges reserved for documentation purposes (RFC 5737). Note, that I'm not worried about a customer's network blocking these IPs, as this is only internal to our server (packets get NATted before leaving the box). However this does seem more unorthodox than just sticking with the default 192.168.122.x/24 subnet.

Read the article

VLAN across a router to give wireless access to remote sites?

- by Don

I've been looking online for this answer, but getting conflicting information. I was under the impression that you couldn't use a VLAN across a router, but maybe it's possible (according to some documentation I see online)? I was hoping someone could clear it up for me. Here's what I'm working with: We have a remote site with a handful of users. We recently gave them an access point (Cisco 1142n) for internal wireless. It's plugged into a switch and working fine (getting IPs from the same DHCP scope as the wired users are getting). Private wireless is set on VL50. At the home office we have private wireless for our internal network working and on VL50, with a test VLAN setup for VL60, which points to our DSL line for the time being. Both private and public wireless works fine internally (not crossing a router). VL50 is named the same at both sites for consistency in naming. If we wanted to give the remote site access to the public wireless (VL60), would that be possible across the routers? For more information, currently the site is connected to the home office via a T1 connection, Cisco routers on both ends. I didn't think it was possible due to the nature of VLANS being layer 2. But, I am from from an expert on this and would appreciate any instruction as to the actual truth of the matter. The end result I'm going for is, how to get our remote sites access to a public (outside) connection along with their private connection, without actually having a DSL (or similar type line) dropped at their location? Thanks in advance for your thoughts.

Read the article

Linux Server partitioning

- by user1717735

There's a lot of infos about this out there, but there's also a lot of contradictory infos… That's why i need some advices about it. So far, on the servers i had home for test (or even "home production") purposes i didn't really care about partitioning and i configured all in / + a swap partition, over RAID 0. Nevertheless, this pattern can't apply to production servers. I have found a good starting point here, but also it depends on what the servers will be used for… So basically, i have a server on which there will be apache, php, mysql. It will have to handle file uploads (up to 2GB) and has 2*2TB hard drive. I plan to set : / 100GB, /var 1000GB (apache files and mysql files will be here), /tmp 800GB (handles the php tmp file) /home 96GB swap 4GB All of this if of course over RAID 1. But actually, it's not a big deal if I lose data being uploaded, so would it be interesting mounting /tmp over raid 0 while maintaining the rest over raid 1? Sounds complicated…

Read the article

Mysql Replication out of sync? What commands do I run to sync it back up?

- by Alex

I have a master-master replication system. However, due to an auto-increment issue, I got an error in replication...and it stopped replicating. Someone told me to do: stop slave; SET GLOBAL SQL_SLAVE_SKIP_COUNTER = 1; start slave; It didn't work. Then they told me to do: SET GLOBAL SQL_SLAVE_SKIP_COUNTER = 2; It didn't work. Then to test it out, I did: SET GLOBAL SQL_SLAVE_SKIP_COUNTER = 99999; It starts, but it is not updating. I created a table on DB1...and it is not showing up on DB2... Below are the SHOW STATUS for both my DB1 and DB2 (I hit them together): mysql> show master status\G *************************** 1. row *************************** File: mysql-bin.000605 Position: 2019727 Binlog_Do_DB: Binlog_Ignore_DB: 1 row in set (0.00 sec) mysql> show slave status\G; *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: Master_User: Master_Port: Connect_Retry: 60 Master_Log_File: mysql-bin.000605 Read_Master_Log_Pos: 2008810 Relay_Log_File: mysqld-relay-bin.001731 Relay_Log_Pos: 10176595 Relay_Master_Log_File: mysql-bin.000470 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 4255373725 Exec_Master_Log_Pos: 10176458 Relay_Log_Space: 135062517347 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 1376343 1 row in set (0.00 sec) How do I fix it so that they sync back up again? Thank you.

Read the article

some HTTPS sites getting blocked on one machine in network

- by shadowfoxmi

I have a few computers connected to the internet via a router. I have been having some trouble with this one Windows 7 desktop. I can browse most of the sites without any trouble but some sites where the sign in page switches to a secure connection (https), the page does not load. It's not all of the sites though. I'm able to sign into gmail and a few other services that I know use https . The sites I'm having trouble with; yahoo's sign in page and the one that I have been using to test across different systems, http://iforgot.apple.com (which switchs to https) ;this particular site, i can access from other computers on the network and my phone. I only have windows firewall running and AVG. I even tried to stopping windows firewall but it did not help. Everything was fine last week. All I have installed in the past week is VOIP softwares namely skype, ooVoo and windows live messenger. I'm not sure how to find out what's being blocked and why and how to unblock it? Any suggestions would be greatly appreciated.

Read the article

Copy past speed very slow for a large number of files on Windows [closed]

- by Arno2501

I've run the following test I've created a folder containing 15'000 files of 400 bytes using this batch : @ECHO off SET times=15000 FOR /L %%i IN (1,1,%times%) DO ( fsutil file createnew filename%%i.txt 400 ) then I copy past it on my Windows Computer using this command : robocopy LargeNumberOfFiles\ LargeNumberOfFiles2\ After it has completed I can see that the transfer rate was 915810 Bytes/sec this is less than 1 MB/s. It took me several seconds to copy 7 MBytes Please note that this is very slow. I've tried the same with a folder with a single file of 50 Mbytes and the transfer rate is 1219512195 Bytes/sec. (yeah GB/s) instantaneous. Why copying large number of files take so much time - ressources on a windows filesystem ? Please note that I've tried to do the same on a linux system which runs on the same computer in a virtual machine (vmware player) with ext3 filesystem. I use the cp command and the copy is instantaneous ! Please also note the following : no antivirus I've tested that behaviour on multiple windows computers (always ntfs) i always get comparable results (transfer rate under 1MB/s avg 7-8 seconds to copy 7 MBytes) I've tested on multiple linux ext3 system the copy is always instantaneous for that amount (15000 files of 400 bytes) The question is about understanding what makes windows filesystem so slow to copy large number of files compared to a linux one for instance.

Read the article

Reporting SQL Vulnerability [migrated]

- by Ciaran87Bel

My first post here so i'll hopefully keep it simple. I have just finished building a CMS targeted at a certain industry and built a test site to see how everything works. Anyway I wrote a program to check for sql injection vulnerabilities and the program followed a blog link to an external website. The program discovered that the external site had a massive vulnerability that left it open to practically anyone who could then access every bit of data on their MYSQL Server and run queries etc. The thing is this external site is the brand leader in their industry and do millions upon millions of sales per annum. I have tried contacting them to let them know and even went as far as contacting the company that built their platform but I was pretty much brushed off and haven't heard back from them. Their database would contain the details of hundreds of thousands of customers and all their data. I could easily make myself site admin etc in a few seconds but they won't listen to me even though I have offered to share the vulnerability with them and help in anyway I can. Is there anything else I can do because it is one of the biggest security risks I have ever personally come across. Is there any other steps I should take to report this? Thanks

Read the article

Creating a file upload template in Doctrine ORM

- by balupton

Hey all. I'm using Doctrine 1.2 as my ORM for a Zend Framework Project. I have defined the following Model for a File. File: columns: id: primary: true type: integer(4) unsigned: true code: type: string(255) unique: true notblank: true path: type: string(255) notblank: true size: type: integer(4) type: type: enum values: [file,document,image,video,audio,web,application,archive] default: unknown notnull: true mimetype: type: string(20) notnull: true width: type: integer(2) unsigned: true height: type: integer(2) unsigned: true Now here is the File Model php class (just skim through for now): <?php /** * File * * This class has been auto-generated by the Doctrine ORM Framework * * @package ##PACKAGE## * @subpackage ##SUBPACKAGE## * @author ##NAME## <##EMAIL##> * @version SVN: $Id: Builder.php 6365 2009-09-15 18:22:38Z jwage $ */ class File extends BaseFile { public function setUp ( ) { $this->hasMutator('file', 'setFile'); parent::setUp(); } public function setFile ( $file ) { global $Application; // Configuration $config = array(); $config['bal'] = $Application->getOption('bal'); // Check the file if ( !empty($file['error']) ) { $error = $file['error']; switch ( $file['error'] ) { case UPLOAD_ERR_INI_SIZE : $error = 'ini_size'; break; case UPLOAD_ERR_FORM_SIZE : $error = 'form_size'; break; case UPLOAD_ERR_PARTIAL : $error = 'partial'; break; case UPLOAD_ERR_NO_FILE : $error = 'no_file'; break; case UPLOAD_ERR_NO_TMP_DIR : $error = 'no_tmp_dir'; break; case UPLOAD_ERR_CANT_WRITE : $error = 'cant_write'; break; default : $error = 'unknown'; break; } throw new Doctrine_Exception('error-application-file-' . $error); return false; } if ( empty($file['tmp_name']) || !is_uploaded_file($file['tmp_name']) ) { throw new Doctrine_Exception('error-application-file-invalid'); return false; } // Prepare config $file_upload_path = realpath($config['bal']['files']['upload_path']) . DIRECTORY_SEPARATOR; // Prepare file $filename = $file['name']; $file_old_path = $file['tmp_name']; $file_new_path = $file_upload_path . $filename; $exist_attempt = 0; while ( file_exists($file_new_path) ) { // File already exists // Pump exist attempts ++$exist_attempt; // Add the attempt to the end of the file $file_new_path = $file_upload_path . get_filename($filename,false) . $exist_attempt . get_extension($filename); } // Move file $success = move_uploaded_file($file_old_path, $file_new_path); if ( !$success ) { throw new Doctrine_Exception('Unable to upload the file.'); return false; } // Secure $file_path = realpath($file_new_path); $file_size = filesize($file_path); $file_mimetype = get_mime_type($file_path); $file_type = get_filetype($file_path); // Apply $this->path = $file_path; $this->size = $file_size; $this->mimetype = $file_mimetype; $this->type = $file_type; // Apply: Image if ( $file_type === 'image' ) { $image_dimensions = image_dimensions($file_path); if ( !empty($image_dimensions) ) { // It is not a image we can modify $this->width = 0; $this->height = 0; } else { $this->width = $image_dimensions['width']; $this->height = $image_dimensions['height']; } } // Done return true; } /** * Download the File * @return */ public function download ( ) { global $Application; // File path $file_upload_path = realpath($config['bal']['files']['upload_path']) . DIRECTORY_SEPARATOR; $file_path = $file_upload_path . $this->file_path; // Output result and download become_file_download($file_path, null, null); die(); } public function postDelete ( $Event ) { global $Application; // Prepare $Invoker = $Event->getInvoker(); // Configuration $config = array(); $config['bal'] = $Application->getOption('bal'); // File path $file_upload_path = realpath($config['bal']['files']['upload_path']) . DIRECTORY_SEPARATOR; $file_path = $file_upload_path . $this->file_path; // Delete the file unlink($file_path); // Done return true; } } What I am hoping to accomplish is so that the above custom functionality within my model file can be turned into a validator, template, or something along the lines. So hopefully I can do something like: File: actAs: BalFile: columns: id: primary: true type: integer(4) unsigned: true code: type: string(255) unique: true notblank: true path: type: string(255) notblank: true size: type: integer(4) type: type: enum values: [file,document,image,video,audio,web,application,archive] default: unknown notnull: true mimetype: type: string(20) notnull: true width: type: integer(2) unsigned: true height: type: integer(2) unsigned: true I'm hoping for a validator so that say if I do $File->setFile($_FILE['uploaded_file']); It will provide a validation error, except in all the doctrine documentation it has little on custom validators, especially in the contect of "virtual" fields. So in summary, my question is: How earth can I go about making a template/extension to porting this functionality? I have tried before with templates but always gave up after a day :/ If you could take the time to port the above I would greatly appreciate it.

Read the article

How to optimize dynamic programming?

- by Chan

Problem A number is called lucky if the sum of its digits, as well as the sum of the squares of its digits is a prime number. How many numbers between A and B are lucky? Input: The first line contains the number of test cases T. Each of the next T lines contains two integers, A and B. Output: Output T lines, one for each case containing the required answer for the corresponding case. Constraints: 1 <= T <= 10000 1 <= A <= B <= 10^18 Sample Input: 2 1 20 120 130 Sample Output: 4 1 Explanation: For the first case, the lucky numbers are 11, 12, 14, 16. For the second case, the only lucky number is 120. The problem is quite simple if we use brute force, however the running time is so critical that my program failed most test cases. My current idea is to use dynamic programming by storing the previous sum in a temporary array, so for example: sum_digits(10) = 1 -> sum_digits(11) = sum_digits(10) + 1 The same idea is applied for sum square but with counter equals to odd numbers. Unfortunately, it still failed 9 of 10 test cases which makes me think there must be a better way to solve it. Any idea would be greatly appreciated. #include <iostream> #include <vector> #include <string> #include <algorithm> #include <unordered_map> #include <unordered_set> #include <cmath> #include <cassert> #include <bitset> using namespace std; bool prime_table[1540] = { 0, 0, 1, 1, 0, 1, 0, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0 }; unsigned num_digits(long long i) { return i > 0 ? (long) log10 ((double) i) + 1 : 1; } void get_sum_and_sum_square_digits(long long n, int& sum, int& sum_square) { sum = 0; sum_square = 0; int digit; while (n) { digit = n % 10; sum += digit; sum_square += digit * digit; n /= 10; } } void init_digits(long long n, long long previous_sum[], const int size = 18) { int current_no_digits = num_digits(n); int digit; for (int i = 0; i < current_no_digits; ++i) { digit = n % 10; previous_sum[i] = digit; n /= 10; } for (int i = current_no_digits; i <= size; ++i) { previous_sum[i] = 0; } } void display_previous(long long previous[]) { for (int i = 0; i < 18; ++i) { cout << previous[i] << ","; } } int count_lucky_number(long long A, long long B) { long long n = A; long long end = B; int sum = 0; int sum_square = 0; int lucky_counter = 0; get_sum_and_sum_square_digits(n, sum, sum_square); long long sum_counter = sum; long long sum_square_counter = sum_square; if (prime_table[sum_counter] && prime_table[sum_square_counter]) { lucky_counter++; } long long previous_sum[19] = {1}; init_digits(n, previous_sum); while (n < end) { n++; if (n % 100000000000000000 == 0) { previous_sum[17]++; sum_counter = previous_sum[17] + previous_sum[18]; sum_square_counter = previous_sum[17] * previous_sum[17] + previous_sum[18] * previous_sum[18]; previous_sum[16] = 0; previous_sum[15] = 0; previous_sum[14] = 0; previous_sum[13] = 0; previous_sum[12] = 0; previous_sum[11] = 0; previous_sum[10] = 0; previous_sum[9] = 0; previous_sum[8] = 0; previous_sum[7] = 0; previous_sum[6] = 0; previous_sum[5] = 0; previous_sum[4] = 0; previous_sum[3] = 0; previous_sum[2] = 0; previous_sum[1] = 0; previous_sum[0] = 0; } else if (n % 10000000000000000 == 0) { previous_sum[16]++; sum_counter = previous_sum[16] + previous_sum[17] + previous_sum[18]; sum_square_counter = previous_sum[16] * previous_sum[16] + previous_sum[17] * previous_sum[17] + previous_sum[18] * previous_sum[18]; previous_sum[15] = 0; previous_sum[14] = 0; previous_sum[13] = 0; previous_sum[12] = 0; previous_sum[11] = 0; previous_sum[10] = 0; previous_sum[9] = 0; previous_sum[8] = 0; previous_sum[7] = 0; previous_sum[6] = 0; previous_sum[5] = 0; previous_sum[4] = 0; previous_sum[3] = 0; previous_sum[2] = 0; previous_sum[1] = 0; previous_sum[0] = 0; } else if (n % 1000000000000000 == 0) { previous_sum[15]++; sum_counter = previous_sum[15] + previous_sum[16] + previous_sum[17] + previous_sum[18]; sum_square_counter = previous_sum[15] * previous_sum[15] + previous_sum[16] * previous_sum[16] + previous_sum[17] * previous_sum[17] + previous_sum[18] * previous_sum[18]; previous_sum[14] = 0; previous_sum[13] = 0; previous_sum[12] = 0; previous_sum[11] = 0; previous_sum[10] = 0; previous_sum[9] = 0; previous_sum[8] = 0; previous_sum[7] = 0; previous_sum[6] = 0; previous_sum[5] = 0; previous_sum[4] = 0; previous_sum[3] = 0; previous_sum[2] = 0; previous_sum[1] = 0; previous_sum[0] = 0; } else if (n % 100000000000000 == 0) { previous_sum[14]++; sum_counter = previous_sum[14] + previous_sum[15] + previous_sum[16] + previous_sum[17] + previous_sum[18]; sum_square_counter = previous_sum[14] * previous_sum[14] + previous_sum[15] * previous_sum[15] + previous_sum[16] * previous_sum[16] + previous_sum[17] * previous_sum[17] + previous_sum[18] * previous_sum[18]; previous_sum[13] = 0; previous_sum[12] = 0; previous_sum[11] = 0; previous_sum[10] = 0; previous_sum[9] = 0; previous_sum[8] = 0; previous_sum[7] = 0; previous_sum[6] = 0; previous_sum[5] = 0; previous_sum[4] = 0; previous_sum[3] = 0; previous_sum[2] = 0; previous_sum[1] = 0; previous_sum[0] = 0; } else if (n % 10000000000000 == 0) { previous_sum[13]++; sum_counter = previous_sum[13] + previous_sum[14] + previous_sum[15] + previous_sum[16] + previous_sum[17] + previous_sum[18]; sum_square_counter = previous_sum[13] * previous_sum[13] + previous_sum[14] * previous_sum[14] + previous_sum[15] * previous_sum[15] + previous_sum[16] * previous_sum[16] + previous_sum[17] * previous_sum[17] + previous_sum[18] * previous_sum[18]; previous_sum[12] = 0; previous_sum[11] = 0; previous_sum[10] = 0; previous_sum[9] = 0; previous_sum[8] = 0; previous_sum[7] = 0; previous_sum[6] = 0; previous_sum[5] = 0; previous_sum[4] = 0; previous_sum[3] = 0; previous_sum[2] = 0; previous_sum[1] = 0; previous_sum[0] = 0; } else if (n % 1000000000000 == 0) { previous_sum[12]++; sum_counter = previous_sum[12] + previous_sum[13] + previous_sum[14] + previous_sum[15] + previous_sum[16] + previous_sum[17] + previous_sum[18]; sum_square_counter = previous_sum[12] * previous_sum[12] + previous_sum[13] * previous_sum[13] + previous_sum[14] * previous_sum[14] + previous_sum[15] * previous_sum[15] + previous_sum[16] * previous_sum[16] + previous_sum[17] * previous_sum[17] + previous_sum[18] * previous_sum[18]; previous_sum[11] = 0; previous_sum[10] = 0; previous_sum[9] = 0; previous_sum[8] = 0; previous_sum[7] = 0; previous_sum[6] = 0; previous_sum[5] = 0; previous_sum[4] = 0; previous_sum[3] = 0; previous_sum[2] = 0; previous_sum[1] = 0; previous_sum[0] = 0; } else if (n % 100000000000 == 0) { previous_sum[11]++; sum_counter = previous_sum[11] + previous_sum[12] + previous_sum[13] + previous_sum[14] + previous_sum[15] + previous_sum[16] + previous_sum[17] + previous_sum[18]; sum_square_counter = previous_sum[11] * previous_sum[11] + previous_sum[12] * previous_sum[12] + previous_sum[13] * previous_sum[13] + previous_sum[14] * previous_sum[14] + previous_sum[15] * previous_sum[15] + previous_sum[16] * previous_sum[16] + previous_sum[17] * previous_sum[17] + previous_sum[18] * previous_sum[18]; previous_sum[10] = 0; previous_sum[9] = 0; previous_sum[8] = 0; previous_sum[7] = 0; previous_sum[6] = 0; previous_sum[5] = 0; previous_sum[4] = 0; previous_sum[3] = 0; previous_sum[2] = 0; previous_sum[1] = 0; previous_sum[0] = 0; } else if (n % 10000000000 == 0) { previous_sum[10]++; sum_counter = previous_sum[10] + previous_sum[11] + previous_sum[12] + previous_sum[13] + previous_sum[14] + previous_sum[15] + previous_sum[16] + previous_sum[17] + previous_sum[18]; sum_square_counter = previous_sum[10] * previous_sum[10] + previous_sum[11] * previous_sum[11] + previous_sum[12] * previous_sum[12] + previous_sum[13] * previous_sum[13] + previous_sum[14] * previous_sum[14] + previous_sum[15] * previous_sum[15] + previous_sum[16] * previous_sum[16] + previous_sum[17] * previous_sum[17] + previous_sum[18] * previous_sum[18]; previous_sum[9] = 0; previous_sum[8] = 0; previous_sum[7] = 0; previous_sum[6] = 0; previous_sum[5] = 0; previous_sum[4] = 0; previous_sum[3] = 0; previous_sum[2] = 0; previous_sum[1] = 0; previous_sum[0] = 0; } else if (n % 1000000000 == 0) { previous_sum[9]++; sum_counter = previous_sum[9] + previous_sum[10] + previous_sum[11] + previous_sum[12] + previous_sum[13] + previous_sum[14] + previous_sum[15] + previous_sum[16] + previous_sum[17] + previous_sum[18]; sum_square_counter = previous_sum[9] * previous_sum[9] + previous_sum[10] * previous_sum[10] + previous_sum[11] * previous_sum[11] + previous_sum[12] * previous_sum[12] + previous_sum[13] * previous_sum[13] + previous_sum[14] * previous_sum[14] + previous_sum[15] * previous_sum[15] + previous_sum[16] * previous_sum[16] + previous_sum[17] * previous_sum[17] + previous_sum[18] * previous_sum[18]; previous_sum[8] = 0; previous_sum[7] = 0; previous_sum[6] = 0; previous_sum[5] = 0; previous_sum[4] = 0; previous_sum[3] = 0; previous_sum[2] = 0; previous_sum[1] = 0; previous_sum[0] = 0; } else if (n % 100000000 == 0) { previous_sum[8]++; sum_counter = previous_sum[8] + previous_sum[9] + previous_sum[10] + previous_sum[11] + previous_sum[12] + previous_sum[13] + previous_sum[14] + previous_sum[15] + previous_sum[16] + previous_sum[17] + previous_sum[18]; sum_square_counter = previous_sum[8] * previous_sum[8] + previous_sum[9] * previous_sum[9] + previous_sum[10] * previous_sum[10] + previous_sum[11] * previous_sum[11] + previous_sum[12] * previous_sum[12] + previous_sum[13] * previous_sum[13] + previous_sum[14] * previous_sum[14] + previous_sum[15] * previous_sum[15] + previous_sum[16] * previous_sum[16] + previous_sum[17] * previous_sum[17] + previous_sum[18] * previous_sum[18]; previous_sum[7] = 0; previous_sum[6] = 0; previous_sum[5] = 0; previous_sum[4] = 0; previous_sum[3] = 0; previous_sum[2] = 0; previous_sum[1] = 0; previous_sum[0] = 0; } else if (n % 10000000 == 0) { previous_sum[7]++; sum_counter = previous_sum[7] + previous_sum[8] + previous_sum[9] + previous_sum[10] + previous_sum[11] + previous_sum[12] + previous_sum[13] + previous_sum[14] + previous_sum[15] + previous_sum[16] + previous_sum[17] + previous_sum[18]; sum_square_counter = previous_sum[7] * previous_sum[7] + previous_sum[8] * previous_sum[8] + previous_sum[9] * previous_sum[9] + previous_sum[10] * previous_sum[10] + previous_sum[11] * previous_sum[11] + previous_sum[12] * previous_sum[12] + previous_sum[13] * previous_sum[13] + previous_sum[14] * previous_sum[14] + previous_sum[15] * previous_sum[15] + previous_sum[16] * previous_sum[16] + previous_sum[17] * previous_sum[17] + previous_sum[18] * previous_sum[18]; previous_sum[6] = 0; previous_sum[5] = 0; previous_sum[4] = 0; previous_sum[3] = 0; previous_sum[2] = 0; previous_sum[1] = 0; previous_sum[0] = 0; } else if (n % 1000000 == 0) { previous_sum[6]++; sum_counter = previous_sum[6] + previous_sum[7] + previous_sum[8] + previous_sum[9] + previous_sum[10] + previous_sum[11] + previous_sum[12] + previous_sum[13] + previous_sum[14] + previous_sum[15] + previous_sum[16] + previous_sum[17] + previous_sum[18]; sum_square_counter = previous_sum[6] * previous_sum[6] + previous_sum[7] * previous_sum[7] + previous_sum[8] * previous_sum[8] + previous_sum[9] * previous_sum[9] + previous_sum[10] * previous_sum[10] + previous_sum[11] * previous_sum[11] + previous_sum[12] * previous_sum[12] + previous_sum[13] * previous_sum[13] + previous_sum[14] * previous_sum[14] + previous_sum[15] * previous_sum[15] + previous_sum[16] * previous_sum[16] + previous_sum[17] * previous_sum[17] + previous_sum[18] * previous_sum[18]; previous_sum[5] = 0; previous_sum[4] = 0; previous_sum[3] = 0; previous_sum[2] = 0; previous_sum[1] = 0; previous_sum[0] = 0; } else if (n % 100000 == 0) { previous_sum[5]++; sum_counter = previous_sum[5] + previous_sum[6] + previous_sum[7] + previous_sum[8] + previous_sum[9] + previous_sum[10] + previous_sum[11] + previous_sum[12] + previous_sum[13] + previous_sum[14] + previous_sum[15] + previous_sum[16] + previous_sum[17] + previous_sum[18]; sum_square_counter = previous_sum[5] * previous_sum[5] + previous_sum[6] * previous_sum[6] + previous_sum[7] * previous_sum[7] + previous_sum[8] * previous_sum[8] + previous_sum[9] * previous_sum[9] + previous_sum[10] * previous_sum[10] + previous_sum[11] * previous_sum[11] + previous_sum[12] * previous_sum[12] + previous_sum[13] * previous_sum[13] + previous_sum[14] * previous_sum[14] + previous_sum[15] * previous_sum[15] + previous_sum[16] * previous_sum[16] + previous_sum[17] * previous_sum[17] + previous_sum[18] * previous_sum[18]; previous_sum[4] = 0; previous_sum[3] = 0; previous_sum[2] = 0; previous_sum[1] = 0; previous_sum[0] = 0; } else if (n % 10000 == 0) { previous_sum[4]++; sum_counter = previous_sum[4] + previous_sum[5] + previous_sum[6] + previous_sum[7] + previous_sum[8] + previous_sum[9] + previous_sum[10] + previous_sum[11] + previous_sum[12] + previous_sum[13] + previous_sum[14] + previous_sum[15] + previous_sum[16] + previous_sum[17] + previous_sum[18]; sum_square_counter = previous_sum[4] * previous_sum[4] + previous_sum[5] * previous_sum[5] + previous_sum[6] * previous_sum[6] + previous_sum[7] * previous_sum[7] + previous_sum[8] * previous_sum[8] + previous_sum[9] * previous_sum[9] + previous_sum[12] * previous_sum[12] + previous_sum[13] * previous_sum[13] + previous_sum[14] * previous_sum[14] + previous_sum[15] * previous_sum[15] + previous_sum[16] * previous_sum[16] + previous_sum[17] * previous_sum[17] + previous_sum[18] * previous_sum[18]; previous_sum[3] = 0; previous_sum[2] = 0; previous_sum[1] = 0; previous_sum[0] = 0; } else if (n % 1000 == 0) { previous_sum[3]++; sum_counter = previous_sum[3] + previous_sum[4] + previous_sum[5] + previous_sum[6] + previous_sum[7] + previous_sum[8] + previous_sum[9] + previous_sum[10] + previous_sum[11] + previous_sum[12] + previous_sum[13] + previous_sum[14] + previous_sum[15] + previous_sum[16] + previous_sum[17] + previous_sum[18]; sum_square_counter = previous_sum[3] * previous_sum[3] + previous_sum[4] * previous_sum[4] + previous_sum[5] * previous_sum[5] + previous_sum[6] * previous_sum[6] + previous_sum[7] * previous_sum[7] + previous_sum[8] * previous_sum[8] + previous_sum[9] * previous_sum[9] + previous_sum[10] * previous_sum[10] + previous_sum[11] * previous_sum[11] + previous_sum[12] * previous_sum[12] + previous_sum[13] * previous_sum[13] + previous_sum[14] * previous_sum[14] + previous_sum[15] * previous_sum[15] + previous_sum[16] * previous_sum[16] + previous_sum[17] * previous_sum[17] + previous_sum[18] * previous_sum[18]; previous_sum[2] = 0; previous_sum[1] = 0; previous_sum[0] = 0; } else if (n % 100 == 0) { previous_sum[2]++; sum_counter = previous_sum[2] + previous_sum[3] + previous_sum[4] + previous_sum[5] + previous_sum[6] + previous_sum[7] + previous_sum[8] + previous_sum[9] + previous_sum[10] + previous_sum[11] + previous_sum[12] + previous_sum[13] + previous_sum[14] + previous_sum[15] + previous_sum[16] + previous_sum[17] + previous_sum[18]; sum_square_counter = previous_sum[2] * previous_sum[2] + previous_sum[3] * previous_sum[3] + previous_sum[4] * previous_sum[4] + previous_sum[5] * previous_sum[5] + previous_sum[6] * previous_sum[6] + previous_sum[7] * previous_sum[7] + previous_sum[8] * previous_sum[8] + previous_sum[9] * previous_sum[9] + previous_sum[10] * previous_sum[10] + previous_sum[11] * previous_sum[11] + previous_sum[12] * previous_sum[12] + previous_sum[13] * previous_sum[13] + previous_sum[14] * previous_sum[14] + previous_sum[15] * previous_sum[15] + previous_sum[16] * previous_sum[16] + previous_sum[17] * previous_sum[17] + previous_sum[18] * previous_sum[18]; previous_sum[1] = 0; previous_sum[0] = 0; } else if (n % 10 == 0) { previous_sum[1]++; sum_counter = previous_sum[1] + previous_sum[2] + previous_sum[3] + previous_sum[4] + previous_sum[5] + previous_sum[6] + previous_sum[7] + previous_sum[8] + previous_sum[9] + previous_sum[10] + previous_sum[11] + previous_sum[12] + previous_sum[13] + previous_sum[14] + previous_sum[15] + previous_sum[16] + previous_sum[17] + previous_sum[18]; sum_square_counter = previous_sum[1] * previous_sum[1] + previous_sum[2] * previous_sum[2] + previous_sum[3] * previous_sum[3] + previous_sum[4] * previous_sum[4] + previous_sum[5] * previous_sum[5] + previous_sum[6] * previous_sum[6] + previous_sum[7] * previous_sum[7] + previous_sum[8] * previous_sum[8] + previous_sum[9] * previous_sum[9] + previous_sum[10] * previous_sum[10] + previous_sum[11] * previous_sum[11] + previous_sum[12] * previous_sum[12] + previous_sum[13] * previous_sum[13] + previous_sum[14] * previous_sum[14] + previous_sum[15] * previous_sum[15] + previous_sum[16] * previous_sum[16] + previous_sum[17] * previous_sum[17] + previous_sum[18] * previous_sum[18]; previous_sum[0] = 0; } else { sum_counter++; sum_square_counter += ((n - 1) % 10) * 2 + 1; } // get_sum_and_sum_square_digits(n, sum, sum_square); // assert(sum == sum_counter && sum_square == sum_square_counter); if (prime_table[sum_counter] && prime_table[sum_square_counter]) { lucky_counter++; } } return lucky_counter; } void inout_lucky_numbers() { int n; cin >> n; long long a; long long b; while (n--) { cin >> a >> b; cout << count_lucky_number(a, b) << endl; } } int main() { inout_lucky_numbers(); return 0; }

Read the article

Service Discovery in WCF 4.0 – Part 1

- by Shaun

When designing a service oriented architecture (SOA) system, there will be a lot of services with many service contracts, endpoints and behaviors. Besides the client calling the service, in a large distributed system a service may invoke other services. In this case, one service might need to know the endpoints it invokes. This might not be a problem in a small system. But when you have more than 10 services this might be a problem. For example in my current product, there are around 10 services, such as the user authentication service, UI integration service, location service, license service, device monitor service, event monitor service, schedule job service, accounting service, player management service, etc.. Benefit of Discovery Service Since almost all my services need to invoke at least one other service. This would be a difficult task to make sure all services endpoints are configured correctly in every service. And furthermore, it would be a nightmare when a service changed its endpoint at runtime. Hence, we need a discovery service to remove the dependency (configuration dependency). A discovery service plays as a service dictionary which stores the relationship between the contracts and the endpoints for every service. By using the discovery service, when service X wants to invoke service Y, it just need to ask the discovery service where is service Y, then the discovery service will return all proper endpoints of service Y, then service X can use the endpoint to send the request to service Y. And when some services changed their endpoint address, all need to do is to update its records in the discovery service then all others will know its new endpoint. In WCF 4.0 Discovery it supports both managed proxy discovery mode and ad-hoc discovery mode. In ad-hoc mode there is no standalone discovery service. When a client wanted to invoke a service, it will broadcast an message (normally in UDP protocol) to the entire network with the service match criteria. All services which enabled the discovery behavior will receive this message and only those matched services will send their endpoint back to the client. The managed proxy discovery service works as I described above. In this post I will only cover the managed proxy mode, where there’s a discovery service. For more information about the ad-hoc mode please refer to the MSDN. Service Announcement and Probe The main functionality of discovery service should be return the proper endpoint addresses back to the service who is looking for. In most cases the consume service (as a client) will send the contract which it wanted to request to the discovery service. And then the discovery service will find the endpoint and respond. Sometimes the contract and endpoint are not enough. It also contains versioning, extensions attributes. This post I will only cover the case includes contract and endpoint. When a client (or sometimes a service who need to invoke another service) need to connect to a target service, it will firstly request the discovery service through the “Probe” method with the criteria. Basically the criteria contains the contract type name of the target service. Then the discovery service will search its endpoint repository by the criteria. The repository might be a database, a distributed cache or a flat XML file. If it matches, the discovery service will grab the endpoint information (it’s called discovery endpoint metadata in WCF) and send back. And this is called “Probe”. Finally the client received the discovery endpoint metadata and will use the endpoint to connect to the target service. Besides the probe, discovery service should take the responsible to know there is a new service available when it goes online, as well as stopped when it goes offline. This feature is named “Announcement”. When a service started and stopped, it will announce to the discovery service. So the basic functionality of a discovery service should includes: 1, An endpoint which receive the service online message, and add the service endpoint information in the discovery repository. 2, An endpoint which receive the service offline message, and remove the service endpoint information from the discovery repository. 3, An endpoint which receive the client probe message, and return the matches service endpoints, and return the discovery endpoint metadata. WCF 4.0 discovery service just covers all these features in it's infrastructure classes. Discovery Service in WCF 4.0 WCF 4.0 introduced a new assembly named System.ServiceModel.Discovery which has all necessary classes and interfaces to build a WS-Discovery compliant discovery service. It supports ad-hoc and managed proxy modes. For the case mentioned in this post, what we need to build is a standalone discovery service, which is the managed proxy discovery service mode. To build a managed discovery service in WCF 4.0 just create a new class inherits from the abstract class System.ServiceModel.Discovery.DiscoveryProxy. This class implemented and abstracted the procedures of service announcement and probe. And it exposes 8 abstract methods where we can implement our own endpoint register, unregister and find logic. These 8 methods are asynchronized, which means all invokes to the discovery service are asynchronously, for better service capability and performance. 1, OnBeginOnlineAnnouncement, OnEndOnlineAnnouncement: Invoked when a service sent the online announcement message. We need to add the endpoint information to the repository in this method. 2, OnBeginOfflineAnnouncement, OnEndOfflineAnnouncement: Invoked when a service sent the offline announcement message. We need to remove the endpoint information from the repository in this method. 3, OnBeginFind, OnEndFind: Invoked when a client sent the probe message that want to find the service endpoint information. We need to look for the proper endpoints by matching the client’s criteria through the repository in this method. 4, OnBeginResolve, OnEndResolve: Invoked then a client sent the resolve message. Different from the find method, when using resolve method the discovery service will return the exactly one service endpoint metadata to the client. In our example we will NOT implement this method. Let’s create our own discovery service, inherit the base System.ServiceModel.Discovery.DiscoveryProxy. We also need to specify the service behavior in this class. Since the build-in discovery service host class only support the singleton mode, we must set its instance context mode to single. 1: using System; 2: using System.Collections.Generic; 3: using System.Linq; 4: using System.Text; 5: using System.ServiceModel.Discovery; 6: using System.ServiceModel; 7: 8: namespace Phare.Service 9: { 10: [ServiceBehavior(InstanceContextMode = InstanceContextMode.Single, ConcurrencyMode = ConcurrencyMode.Multiple)] 11: public class ManagedProxyDiscoveryService : DiscoveryProxy 12: { 13: protected override IAsyncResult OnBeginFind(FindRequestContext findRequestContext, AsyncCallback callback, object state) 14: { 15: throw new NotImplementedException(); 16: } 17: 18: protected override IAsyncResult OnBeginOfflineAnnouncement(DiscoveryMessageSequence messageSequence, EndpointDiscoveryMetadata endpointDiscoveryMetadata, AsyncCallback callback, object state) 19: { 20: throw new NotImplementedException(); 21: } 22: 23: protected override IAsyncResult OnBeginOnlineAnnouncement(DiscoveryMessageSequence messageSequence, EndpointDiscoveryMetadata endpointDiscoveryMetadata, AsyncCallback callback, object state) 24: { 25: throw new NotImplementedException(); 26: } 27: 28: protected override IAsyncResult OnBeginResolve(ResolveCriteria resolveCriteria, AsyncCallback callback, object state) 29: { 30: throw new NotImplementedException(); 31: } 32: 33: protected override void OnEndFind(IAsyncResult result) 34: { 35: throw new NotImplementedException(); 36: } 37: 38: protected override void OnEndOfflineAnnouncement(IAsyncResult result) 39: { 40: throw new NotImplementedException(); 41: } 42: 43: protected override void OnEndOnlineAnnouncement(IAsyncResult result) 44: { 45: throw new NotImplementedException(); 46: } 47: 48: protected override EndpointDiscoveryMetadata OnEndResolve(IAsyncResult result) 49: { 50: throw new NotImplementedException(); 51: } 52: } 53: } Then let’s implement the online, offline and find methods one by one. WCF discovery service gives us full flexibility to implement the endpoint add, remove and find logic. For the demo purpose we will use an internal dictionary to store the services’ endpoint metadata. In the next post we will see how to serialize and store these information in database. Define a concurrent dictionary inside the service class since our it will be used in the multiple threads scenario. 1: [ServiceBehavior(InstanceContextMode = InstanceContextMode.Single, ConcurrencyMode = ConcurrencyMode.Multiple)] 2: public class ManagedProxyDiscoveryService : DiscoveryProxy 3: { 4: private ConcurrentDictionary<EndpointAddress, EndpointDiscoveryMetadata> _services; 5: 6: public ManagedProxyDiscoveryService() 7: { 8: _services = new ConcurrentDictionary<EndpointAddress, EndpointDiscoveryMetadata>(); 9: } 10: } Then we can simply implement the logic of service online and offline. 1: protected override IAsyncResult OnBeginOnlineAnnouncement(DiscoveryMessageSequence messageSequence, EndpointDiscoveryMetadata endpointDiscoveryMetadata, AsyncCallback callback, object state) 2: { 3: _services.AddOrUpdate(endpointDiscoveryMetadata.Address, endpointDiscoveryMetadata, (key, value) => endpointDiscoveryMetadata); 4: return new OnOnlineAnnouncementAsyncResult(callback, state); 5: } 6: 7: protected override void OnEndOnlineAnnouncement(IAsyncResult result) 8: { 9: OnOnlineAnnouncementAsyncResult.End(result); 10: } 11: 12: protected override IAsyncResult OnBeginOfflineAnnouncement(DiscoveryMessageSequence messageSequence, EndpointDiscoveryMetadata endpointDiscoveryMetadata, AsyncCallback callback, object state) 13: { 14: EndpointDiscoveryMetadata endpoint = null; 15: _services.TryRemove(endpointDiscoveryMetadata.Address, out endpoint); 16: return new OnOfflineAnnouncementAsyncResult(callback, state); 17: } 18: 19: protected override void OnEndOfflineAnnouncement(IAsyncResult result) 20: { 21: OnOfflineAnnouncementAsyncResult.End(result); 22: } Regards the find method, the parameter FindRequestContext.Criteria has a method named IsMatch, which can be use for us to evaluate which service metadata is satisfied with the criteria. So the implementation of find method would be like this. 1: protected override IAsyncResult OnBeginFind(FindRequestContext findRequestContext, AsyncCallback callback, object state) 2: { 3: _services.Where(s => findRequestContext.Criteria.IsMatch(s.Value)) 4: .Select(s => s.Value) 5: .All(meta => 6: { 7: findRequestContext.AddMatchingEndpoint(meta); 8: return true; 9: }); 10: return new OnFindAsyncResult(callback, state); 11: } 12: 13: protected override void OnEndFind(IAsyncResult result) 14: { 15: OnFindAsyncResult.End(result); 16: } As you can see, we checked all endpoints metadata in repository by invoking the IsMatch method. Then add all proper endpoints metadata into the parameter. Finally since all these methods are asynchronized we need some AsyncResult classes as well. Below are the base class and the inherited classes used in previous methods. 1: using System; 2: using System.Collections.Generic; 3: using System.Linq; 4: using System.Text; 5: using System.Threading; 6: 7: namespace Phare.Service 8: { 9: abstract internal class AsyncResult : IAsyncResult 10: { 11: AsyncCallback callback; 12: bool completedSynchronously; 13: bool endCalled; 14: Exception exception; 15: bool isCompleted; 16: ManualResetEvent manualResetEvent; 17: object state; 18: object thisLock; 19: 20: protected AsyncResult(AsyncCallback callback, object state) 21: { 22: this.callback = callback; 23: this.state = state; 24: this.thisLock = new object(); 25: } 26: 27: public object AsyncState 28: { 29: get 30: { 31: return state; 32: } 33: } 34: 35: public WaitHandle AsyncWaitHandle 36: { 37: get 38: { 39: if (manualResetEvent != null) 40: { 41: return manualResetEvent; 42: } 43: lock (ThisLock) 44: { 45: if (manualResetEvent == null) 46: { 47: manualResetEvent = new ManualResetEvent(isCompleted); 48: } 49: } 50: return manualResetEvent; 51: } 52: } 53: 54: public bool CompletedSynchronously 55: { 56: get 57: { 58: return completedSynchronously; 59: } 60: } 61: 62: public bool IsCompleted 63: { 64: get 65: { 66: return isCompleted; 67: } 68: } 69: 70: object ThisLock 71: { 72: get 73: { 74: return this.thisLock; 75: } 76: } 77: 78: protected static TAsyncResult End<TAsyncResult>(IAsyncResult result) 79: where TAsyncResult : AsyncResult 80: { 81: if (result == null) 82: { 83: throw new ArgumentNullException("result"); 84: } 85: 86: TAsyncResult asyncResult = result as TAsyncResult; 87: 88: if (asyncResult == null) 89: { 90: throw new ArgumentException("Invalid async result.", "result"); 91: } 92: 93: if (asyncResult.endCalled) 94: { 95: throw new InvalidOperationException("Async object already ended."); 96: } 97: 98: asyncResult.endCalled = true; 99: 100: if (!asyncResult.isCompleted) 101: { 102: asyncResult.AsyncWaitHandle.WaitOne(); 103: } 104: 105: if (asyncResult.manualResetEvent != null) 106: { 107: asyncResult.manualResetEvent.Close(); 108: } 109: 110: if (asyncResult.exception != null) 111: { 112: throw asyncResult.exception; 113: } 114: 115: return asyncResult; 116: } 117: 118: protected void Complete(bool completedSynchronously) 119: { 120: if (isCompleted) 121: { 122: throw new InvalidOperationException("This async result is already completed."); 123: } 124: 125: this.completedSynchronously = completedSynchronously; 126: 127: if (completedSynchronously) 128: { 129: this.isCompleted = true; 130: } 131: else 132: { 133: lock (ThisLock) 134: { 135: this.isCompleted = true; 136: if (this.manualResetEvent != null) 137: { 138: this.manualResetEvent.Set(); 139: } 140: } 141: } 142: 143: if (callback != null) 144: { 145: callback(this); 146: } 147: } 148: 149: protected void Complete(bool completedSynchronously, Exception exception) 150: { 151: this.exception = exception; 152: Complete(completedSynchronously); 153: } 154: } 155: } 1: using System; 2: using System.Collections.Generic; 3: using System.Linq; 4: using System.Text; 5: using System.ServiceModel.Discovery; 6: using Phare.Service; 7: 8: namespace Phare.Service 9: { 10: internal sealed class OnOnlineAnnouncementAsyncResult : AsyncResult 11: { 12: public OnOnlineAnnouncementAsyncResult(AsyncCallback callback, object state) 13: : base(callback, state) 14: { 15: this.Complete(true); 16: } 17: 18: public static void End(IAsyncResult result) 19: { 20: AsyncResult.End<OnOnlineAnnouncementAsyncResult>(result); 21: } 22: 23: } 24: 25: sealed class OnOfflineAnnouncementAsyncResult : AsyncResult 26: { 27: public OnOfflineAnnouncementAsyncResult(AsyncCallback callback, object state) 28: : base(callback, state) 29: { 30: this.Complete(true); 31: } 32: 33: public static void End(IAsyncResult result) 34: { 35: AsyncResult.End<OnOfflineAnnouncementAsyncResult>(result); 36: } 37: } 38: 39: sealed class OnFindAsyncResult : AsyncResult 40: { 41: public OnFindAsyncResult(AsyncCallback callback, object state) 42: : base(callback, state) 43: { 44: this.Complete(true); 45: } 46: 47: public static void End(IAsyncResult result) 48: { 49: AsyncResult.End<OnFindAsyncResult>(result); 50: } 51: } 52: 53: sealed class OnResolveAsyncResult : AsyncResult 54: { 55: EndpointDiscoveryMetadata matchingEndpoint; 56: 57: public OnResolveAsyncResult(EndpointDiscoveryMetadata matchingEndpoint, AsyncCallback callback, object state) 58: : base(callback, state) 59: { 60: this.matchingEndpoint = matchingEndpoint; 61: this.Complete(true); 62: } 63: 64: public static EndpointDiscoveryMetadata End(IAsyncResult result) 65: { 66: OnResolveAsyncResult thisPtr = AsyncResult.End<OnResolveAsyncResult>(result); 67: return thisPtr.matchingEndpoint; 68: } 69: } 70: } Now we have finished the discovery service. The next step is to host it. The discovery service is a standard WCF service. So we can use ServiceHost on a console application, windows service, or in IIS as usual. The following code is how to host the discovery service we had just created in a console application. 1: static void Main(string[] args) 2: { 3: using (var host = new ServiceHost(new ManagedProxyDiscoveryService())) 4: { 5: host.Opened += (sender, e) => 6: { 7: host.Description.Endpoints.All((ep) => 8: { 9: Console.WriteLine(ep.ListenUri); 10: return true; 11: }); 12: }; 13: 14: try 15: { 16: // retrieve the announcement, probe endpoint and binding from configuration 17: var announcementEndpointAddress = new EndpointAddress(ConfigurationManager.AppSettings["announcementEndpointAddress"]); 18: var probeEndpointAddress = new EndpointAddress(ConfigurationManager.AppSettings["probeEndpointAddress"]); 19: var binding = Activator.CreateInstance(Type.GetType(ConfigurationManager.AppSettings["bindingType"], true, true)) as Binding; 20: var announcementEndpoint = new AnnouncementEndpoint(binding, announcementEndpointAddress); 21: var probeEndpoint = new DiscoveryEndpoint(binding, probeEndpointAddress); 22: probeEndpoint.IsSystemEndpoint = false; 23: // append the service endpoint for announcement and probe 24: host.AddServiceEndpoint(announcementEndpoint); 25: host.AddServiceEndpoint(probeEndpoint); 26: 27: host.Open(); 28: 29: Console.WriteLine("Press any key to exit."); 30: Console.ReadKey(); 31: } 32: catch (Exception ex) 33: { 34: Console.WriteLine(ex.ToString()); 35: } 36: } 37: 38: Console.WriteLine("Done."); 39: Console.ReadKey(); 40: } What we need to notice is that, the discovery service needs two endpoints for announcement and probe. In this example I just retrieve them from the configuration file. I also specified the binding of these two endpoints in configuration file as well. 1: <?xml version="1.0"?> 2: <configuration> 3: <startup> 4: <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.0"/> 5: </startup> 6: <appSettings> 7: <add key="announcementEndpointAddress" value="net.tcp://localhost:10010/announcement"/> 8: <add key="probeEndpointAddress" value="net.tcp://localhost:10011/probe"/> 9: <add key="bindingType" value="System.ServiceModel.NetTcpBinding, System.ServiceModel, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089"/> 10: </appSettings> 11: </configuration> And this is the console screen when I ran my discovery service. As you can see there are two endpoints listening for announcement message and probe message. Discoverable Service and Client Next, let’s create a WCF service that is discoverable, which means it can be found by the discovery service. To do so, we need to let the service send the online announcement message to the discovery service, as well as offline message before it shutdown. Just create a simple service which can make the incoming string to upper. The service contract and implementation would be like this. 1: [ServiceContract] 2: public interface IStringService 3: { 4: [OperationContract] 5: string ToUpper(string content); 6: } 1: public class StringService : IStringService 2: { 3: public string ToUpper(string content) 4: { 5: return content.ToUpper(); 6: } 7: } Then host this service in the console application. In order to make the discovery service easy to be tested the service address will be changed each time it’s started. 1: static void Main(string[] args) 2: { 3: var baseAddress = new Uri(string.Format("net.tcp://localhost:11001/stringservice/{0}/", Guid.NewGuid().ToString())); 4: 5: using (var host = new ServiceHost(typeof(StringService), baseAddress)) 6: { 7: host.Opened += (sender, e) => 8: { 9: Console.WriteLine("Service opened at {0}", host.Description.Endpoints.First().ListenUri); 10: }; 11: 12: host.AddServiceEndpoint(typeof(IStringService), new NetTcpBinding(), string.Empty); 13: 14: host.Open(); 15: 16: Console.WriteLine("Press any key to exit."); 17: Console.ReadKey(); 18: } 19: } Currently this service is NOT discoverable. We need to add a special service behavior so that it could send the online and offline message to the discovery service announcement endpoint when the host is opened and closed. WCF 4.0 introduced a service behavior named ServiceDiscoveryBehavior. When we specified the announcement endpoint address and appended it to the service behaviors this service will be discoverable. 1: var announcementAddress = new EndpointAddress(ConfigurationManager.AppSettings["announcementEndpointAddress"]); 2: var announcementBinding = Activator.CreateInstance(Type.GetType(ConfigurationManager.AppSettings["bindingType"], true, true)) as Binding; 3: var announcementEndpoint = new AnnouncementEndpoint(announcementBinding, announcementAddress); 4: var discoveryBehavior = new ServiceDiscoveryBehavior(); 5: discoveryBehavior.AnnouncementEndpoints.Add(announcementEndpoint); 6: host.Description.Behaviors.Add(discoveryBehavior); The ServiceDiscoveryBehavior utilizes the service extension and channel dispatcher to implement the online and offline announcement logic. In short, it injected the channel open and close procedure and send the online and offline message to the announcement endpoint. On client side, when we have the discovery service, a client can invoke a service without knowing its endpoint. WCF discovery assembly provides a class named DiscoveryClient, which can be used to find the proper service endpoint by passing the criteria. In the code below I initialized the DiscoveryClient, specified the discovery service probe endpoint address. Then I created the find criteria by specifying the service contract I wanted to use and invoke the Find method. This will send the probe message to the discovery service and it will find the endpoints back to me. The discovery service will return all endpoints that matches the find criteria, which means in the result of the find method there might be more than one endpoints. In this example I just returned the first matched one back. In the next post I will show how to extend our discovery service to make it work like a service load balancer. 1: static EndpointAddress FindServiceEndpoint() 2: { 3: var probeEndpointAddress = new EndpointAddress(ConfigurationManager.AppSettings["probeEndpointAddress"]); 4: var probeBinding = Activator.CreateInstance(Type.GetType(ConfigurationManager.AppSettings["bindingType"], true, true)) as Binding; 5: var discoveryEndpoint = new DiscoveryEndpoint(probeBinding, probeEndpointAddress); 6: 7: EndpointAddress address = null; 8: FindResponse result = null; 9: using (var discoveryClient = new DiscoveryClient(discoveryEndpoint)) 10: { 11: result = discoveryClient.Find(new FindCriteria(typeof(IStringService))); 12: } 13: 14: if (result != null && result.Endpoints.Any()) 15: { 16: var endpointMetadata = result.Endpoints.First(); 17: address = endpointMetadata.Address; 18: } 19: return address; 20: } Once we probed the discovery service we will receive the endpoint. So in the client code we can created the channel factory from the endpoint and binding, and invoke to the service. When creating the client side channel factory we need to make sure that the client side binding should be the same as the service side. WCF discovery service can be used to find the endpoint for a service contract, but the binding is NOT included. This is because the binding was not in the WS-Discovery specification. In the next post I will demonstrate how to add the binding information into the discovery service. At that moment the client don’t need to create the binding by itself. Instead it will use the binding received from the discovery service. 1: static void Main(string[] args) 2: { 3: Console.WriteLine("Say something..."); 4: var content = Console.ReadLine(); 5: while (!string.IsNullOrWhiteSpace(content)) 6: { 7: Console.WriteLine("Finding the service endpoint..."); 8: var address = FindServiceEndpoint(); 9: if (address == null) 10: { 11: Console.WriteLine("There is no endpoint matches the criteria."); 12: } 13: else 14: { 15: Console.WriteLine("Found the endpoint {0}", address.Uri); 16: 17: var factory = new ChannelFactory<IStringService>(new NetTcpBinding(), address); 18: factory.Opened += (sender, e) => 19: { 20: Console.WriteLine("Connecting to {0}.", factory.Endpoint.ListenUri); 21: }; 22: var proxy = factory.CreateChannel(); 23: using (proxy as IDisposable) 24: { 25: Console.WriteLine("ToUpper: {0} => {1}", content, proxy.ToUpper(content)); 26: } 27: } 28: 29: Console.WriteLine("Say something..."); 30: content = Console.ReadLine(); 31: } 32: } Similarly, the discovery service probe endpoint and binding were defined in the configuration file. 1: <?xml version="1.0"?> 2: <configuration> 3: <startup> 4: <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.0"/> 5: </startup> 6: <appSettings> 7: <add key="announcementEndpointAddress" value="net.tcp://localhost:10010/announcement"/> 8: <add key="probeEndpointAddress" value="net.tcp://localhost:10011/probe"/> 9: <add key="bindingType" value="System.ServiceModel.NetTcpBinding, System.ServiceModel, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089"/> 10: </appSettings> 11: </configuration> OK, now let’s have a test. Firstly start the discovery service, and then start our discoverable service. When it started it will announced to the discovery service and registered its endpoint into the repository, which is the local dictionary. And then start the client and type something. As you can see the client asked the discovery service for the endpoint and then establish the connection to the discoverable service. And more interesting, do NOT close the client console but terminate the discoverable service but press the enter key. This will make the service send the offline message to the discovery service. Then start the discoverable service again. Since we made it use a different address each time it started, currently it should be hosted on another address. If we enter something in the client we could see that it asked the discovery service and retrieve the new endpoint, and connect the the service. Summary In this post I discussed the benefit of using the discovery service and the procedures of service announcement and probe. I also demonstrated how to leverage the WCF Discovery feature in WCF 4.0 to build a simple managed discovery service. For test purpose, in this example I used the in memory dictionary as the discovery endpoint metadata repository. And when finding I also just return the first matched endpoint back. I also hard coded the bindings between the discoverable service and the client. In next post I will show you how to solve the problem mentioned above, as well as some additional feature for production usage. You can download the code here. Hope this helps, Shaun All documents and related graphics, codes are provided "AS IS" without warranty of any kind. Copyright © Shaun Ziyan Xu. This work is licensed under the Creative Commons License.

Read the article

Windows Azure: Import/Export Hard Drives, VM ACLs, Web Sockets, Remote Debugging, Continuous Delivery, New Relic, Billing Alerts and More

- by ScottGu

Two weeks ago we released a giant set of improvements to Windows Azure, as well as a significant update of the Windows Azure SDK. This morning we released another massive set of enhancements to Windows Azure. Today’s new capabilities include: Storage: Import/Export Hard Disk Drives to your Storage Accounts HDInsight: General Availability of our Hadoop Service in the cloud Virtual Machines: New VM Gallery, ACL support for VIPs Web Sites: WebSocket and Remote Debugging Support Notification Hubs: Segmented customer push notification support with tag expressions TFS & GIT: Continuous Delivery Support for Web Sites + Cloud Services Developer Analytics: New Relic support for Web Sites + Mobile Services Service Bus: Support for partitioned queues and topics Billing: New Billing Alert Service that sends emails notifications when your bill hits a threshold you define All of these improvements are now available to use immediately (note that some features are still in preview). Below are more details about them. Storage: Import/Export Hard Disk Drives to Windows Azure I am excited to announce the preview of our new Windows Azure Import/Export Service! The Windows Azure Import/Export Service enables you to move large amounts of on-premises data into and out of your Windows Azure Storage accounts. It does this by enabling you to securely ship hard disk drives directly to our Windows Azure data centers. Once we receive the drives we’ll automatically transfer the data to or from your Windows Azure Storage account. This enables you to import or export massive amounts of data more quickly and cost effectively (and not be constrained by available network bandwidth). Encrypted Transport Our Import/Export service provides built-in support for BitLocker disk encryption – which enables you to securely encrypt data on the hard drives before you send it, and not have to worry about it being compromised even if the disk is lost/stolen in transit (since the content on the transported hard drives is completely encrypted and you are the only one who has the key to it). The drive preparation tool we are shipping today makes setting up bitlocker encryption on these hard drives easy. How to Import/Export your first Hard Drive of Data You can read our Getting Started Guide to learn more about how to begin using the import/export service. You can create import and export jobs via the Windows Azure Management Portal as well as programmatically using our Server Management APIs. It is really easy to create a new import or export job using the Windows Azure Management Portal. Simply navigate to a Windows Azure storage account, and then click the new Import/Export tab now available within it (note: if you don’t have this tab make sure to sign-up for the Import/Export preview): Then click the “Create Import Job” or “Create Export Job” commands at the bottom of it. This will launch a wizard that easily walks you through the steps required: For more comprehensive information about Import/Export, refer to Windows Azure Storage team blog. You can also send questions and comments to the [email protected] email address. We think you’ll find this new service makes it much easier to move data into and out of Windows Azure, and it will dramatically cut down the network bandwidth required when working on large data migration projects. We hope you like it. HDInsight: 100% Compatible Hadoop Service in the Cloud Last week we announced the general availability release of Windows Azure HDInsight. HDInsight is a 100% compatible Hadoop service that allows you to easily provision and manage Hadoop clusters for big data processing in Windows Azure. This release is now live in production, backed by an enterprise SLA, supported 24x7 by Microsoft Support, and is ready to use for production scenarios. HDInsight allows you to use Apache Hadoop tools, such as Pig and Hive, to process large amounts of data in Windows Azure Blob Storage. Because data is stored in Windows Azure Blob Storage, you can choose to dynamically create Hadoop clusters only when you need them, and then shut them down when they are no longer required (since you pay only for the time the Hadoop cluster instances are running this provides a super cost effective way to use them). You can create Hadoop clusters using either the Windows Azure Management Portal (see below) or using our PowerShell and Cross Platform Command line tools: The import/export hard drive support that came out today is a perfect companion service to use with HDInsight – the combination allows you to easily ingest, process and optionally export a limitless amount of data. We’ve also integrated HDInsight with our Business Intelligence tools, so users can leverage familiar tools like Excel in order to analyze the output of jobs. You can find out more about how to get started with HDInsight here. Virtual Machines: VM Gallery Enhancements Today’s update of Windows Azure brings with it a new Virtual Machine gallery that you can use to create new VMs in the cloud. You can launch the gallery by doing New->Compute->Virtual Machine->From Gallery within the Windows Azure Management Portal: The new Virtual Machine Gallery includes some nice enhancements that make it even easier to use: Search: You can now easily search and filter images using the search box in the top-right of the dialog. For example, simply type “SQL” and we’ll filter to show those images in the gallery that contain that substring. Category Tree-view: Each month we add more built-in VM images to the gallery. You can continue to browse these using the “All” view within the VM Gallery – or now quickly filter them using the category tree-view on the left-hand side of the dialog. For example, by selecting “Oracle” in the tree-view you can now quickly filter to see the official Oracle supplied images. MSDN and Supported checkboxes: With today’s update we are also introducing filters that makes it easy to filter out types of images that you may not be interested in. The first checkbox is MSDN: using this filter you can exclude any image that is not part of the Windows Azure benefits for MSDN subscribers (which have highly discounted pricing - you can learn more about the MSDN pricing here). The second checkbox is Supported: this filter will exclude any image that contains prerelease software, so you can feel confident that the software you choose to deploy is fully supported by Windows Azure and our partners. Sort options: We sort gallery images by what we think customers are most interested in, but sometimes you might want to sort using different views. So we’re providing some additional sort options, like “Newest,” to customize the image list for what suits you best. Pricing information: We now provide additional pricing information about images and options on how to cost effectively run them directly within the VM Gallery. The above improvements make it even easier to use the VM Gallery and quickly create launch and run Virtual Machines in the cloud. Virtual Machines: ACL Support for VIPs A few months ago we exposed the ability to configure Access Control Lists (ACLs) for Virtual Machines using Windows PowerShell cmdlets and our Service Management API. With today’s release, you can now configure VM ACLs using the Windows Azure Management Portal as well. You can now do this by clicking the new Manage ACL command in the Endpoints tab of a virtual machine instance: This will enable you to configure an ordered list of permit and deny rules to scope the traffic that can access your VM’s network endpoints. For example, if you were on a virtual network, you could limit RDP access to a Windows Azure virtual machine to only a few computers attached to your enterprise. Or if you weren’t on a virtual network you could alternatively limit traffic from public IPs that can access your workloads: Here is the default behaviors for ACLs in Windows Azure: By default (i.e. no rules specified), all traffic is permitted. When using only Permit rules, all other traffic is denied. When using only Deny rules, all other traffic is permitted. When there is a combination of Permit and Deny rules, all other traffic is denied. Lastly, remember that configuring endpoints does not automatically configure them within the VM if it also has firewall rules enabled at the OS level. So if you create an endpoint using the Windows Azure Management Portal, Windows PowerShell, or REST API, be sure to also configure your guest VM firewall appropriately as well. Web Sites: Web Sockets Support With today’s release you can now use Web Sockets with Windows Azure Web Sites. This feature enables you to easily integrate real-time communication scenarios within your web based applications, and is available at no extra charge (it even works with the free tier). Higher level programming libraries like SignalR and socket.io are also now supported with it. You can enable Web Sockets support on a web site by navigating to the Configure tab of a Web Site, and by toggling Web Sockets support to “on”: Once Web Sockets is enabled you can start to integrate some really cool scenarios into your web applications. Check out the new SignalR documentation hub on www.asp.net to learn more about some of the awesome scenarios you can do with it. Web Sites: Remote Debugging Support The Windows Azure SDK 2.2 we released two weeks ago introduced remote debugging support for Windows Azure Cloud Services. With today’s Windows Azure release we are extending this remote debugging support to also work with Windows Azure Web Sites. With live, remote debugging support inside of Visual Studio, you are able to have more visibility than ever before into how your code is operating live in Windows Azure. It is now super easy to attach the debugger and quickly see what is going on with your application in the cloud. Remote Debugging of a Windows Azure Web Site using VS 2013 Enabling the remote debugging of a Windows Azure Web Site using VS 2013 is really easy. Start by opening up your web application’s project within Visual Studio. Then navigate to the “Server Explorer” tab within Visual Studio, and click on the deployed web-site you want to debug that is running within Windows Azure using the Windows Azure->Web Sites node in the Server Explorer. Then right-click and choose the “Attach Debugger” option on it: When you do this Visual Studio will remotely attach the debugger to the Web Site running within Windows Azure. The debugger will then stop the web site’s execution when it hits any break points that you have set within your web application’s project inside Visual Studio. For example, below I set a breakpoint on the “ViewBag.Message” assignment statement within the HomeController of the standard ASP.NET MVC project template. When I hit refresh on the “About” page of the web site within the browser, the breakpoint was triggered and I am now able to debug the app remotely using Visual Studio: Note above how we can debug variables (including autos/watchlist/etc), as well as use the Immediate and Command Windows. In the debug session above I used the Immediate Window to explore some of the request object state, as well as to dynamically change the ViewBag.Message property. When we click the the “Continue” button (or press F5) the app will continue execution and the Web Site will render the content back to the browser. This makes it super easy to debug web apps remotely. Tips for Better Debugging To get the best experience while debugging, we recommend publishing your site using the Debug configuration within Visual Studio’s Web Publish dialog. This will ensure that debug symbol information is uploaded to the Web Site which will enable a richer debug experience within Visual Studio. You can find this option on the Web Publish dialog on the Settings tab: When you ultimately deploy/run the application in production we recommend using the “Release” configuration setting – the release configuration is memory optimized and will provide the best production performance. To learn more about diagnosing and debugging Windows Azure Web Sites read our new Troubleshooting Windows Azure Web Sites in Visual Studio guide. Notification Hubs: Segmented Push Notification support with tag expressions In August we announced the General Availability of Windows Azure Notification Hubs - a powerful Mobile Push Notifications service that makes it easy to send high volume push notifications with low latency from any mobile app back-end. Notification hubs can be used with any mobile app back-end (including ones built using our Mobile Services capability) and can also be used with back-ends that run in the cloud as well as on-premises. Beginning with the initial release, Notification Hubs allowed developers to send personalized push notifications to both individual users as well as groups of users by interest, by associating their devices with tags representing the logical target of the notification. For example, by registering all devices of customers interested in a favorite MLB team with a corresponding tag, it is possible to broadcast one message to millions of Boston Red Sox fans and another message to millions of St. Louis Cardinals fans with a single API call respectively. New support for using tag expressions to enable advanced customer segmentation With today’s release we are adding support for even more advanced customer targeting. You can now identify customers that you want to send push notifications to by defining rich tag expressions. With tag expressions, you can now not only broadcast notifications to Boston Red Sox fans, but take that segmenting a step farther and reach more granular segments. This opens up a variety of scenarios, for example: Offers based on multiple preferences—e.g. send a game day vegetarian special to users tagged as both a Boston Red Sox fan AND a vegetarian Push content to multiple segments in a single message—e.g. rain delay information only to users who are tagged as either a Boston Red Sox fan OR a St. Louis Cardinal fan Avoid presenting subsets of a segment with irrelevant content—e.g. season ticket availability reminder to users who are tagged as a Boston Red Sox fan but NOT also a season ticket holder To illustrate with code, consider a restaurant chain app that sends an offer related to a Red Sox vs Cardinals game for users in Boston. Devices can be tagged by your app with location tags (e.g. “Loc:Boston”) and interest tags (e.g. “Follows:RedSox”, “Follows:Cardinals”), and then a notification can be sent by your back-end to “(Follows:RedSox || Follows:Cardinals) && Loc:Boston” in order to deliver an offer to all devices in Boston that follow either the RedSox or the Cardinals. This can be done directly in your server backend send logic using the code below: var notification = new WindowsNotification(messagePayload); hub.SendNotificationAsync(notification, "(Follows:RedSox || Follows:Cardinals) && Loc:Boston"); In your expressions you can use all Boolean operators: AND (&&), OR (||), and NOT (!). Some other cool use cases for tag expressions that are now supported include: Social: To “all my group except me” - group:id && !user:id Events: Touchdown event is sent to everybody following either team or any of the players involved in the action: Followteam:A || Followteam:B || followplayer:1 || followplayer:2 … Hours: Send notifications at specific times. E.g. Tag devices with time zone and when it is 12pm in Seattle send to: GMT8 && follows:thaifood Versions and platforms: Send a reminder to people still using your first version for Android - version:1.0 && platform:Android For help on getting started with Notification Hubs, visit the Notification Hub documentation center. Then download the latest NuGet package (or use the Notification Hubs REST APIs directly) to start sending push notifications using tag expressions. They are really powerful and enable a bunch of great new scenarios. TFS & GIT: Continuous Delivery Support for Web Sites + Cloud Services With today’s Windows Azure release we are making it really easy to enable continuous delivery support with Windows Azure and Team Foundation Services. Team Foundation Services is a cloud based offering from Microsoft that provides integrated source control (with both TFS and Git support), build server, test execution, collaboration tools, and agile planning support. It makes it really easy to setup a team project (complete with automated builds and test runners) in the cloud, and it has really rich integration with Visual Studio. With today’s Windows Azure release it is now really easy to enable continuous delivery support with both TFS and Git based repositories hosted using Team Foundation Services. This enables a workflow where when code is checked in, built successfully on an automated build server, and all tests pass on it – I can automatically have the app deployed on Windows Azure with zero manual intervention or work required. The below screen-shots demonstrate how to quickly setup a continuous delivery workflow to Windows Azure with a Git-based ASP.NET MVC project hosted using Team Foundation Services. Enabling Continuous Delivery to Windows Azure with Team Foundation Services The project I’m going to enable continuous delivery with is a simple ASP.NET MVC project whose source code I’m hosting using Team Foundation Services. I did this by creating a “SimpleContinuousDeploymentTest” repository there using Git – and then used the new built-in Git tooling support within Visual Studio 2013 to push the source code to it. Below is a screen-shot of the Git repository hosted within Team Foundation Services: I can access the repository within Visual Studio 2013 and easily make commits with it (as well as branch, merge and do other tasks). Using VS 2013 I can also setup automated builds to take place in the cloud using Team Foundation Services every time someone checks in code to the repository: The cool thing about this is that I don’t have to buy or rent my own build server – Team Foundation Services automatically maintains its own build server farm and can automatically queue up a build for me (for free) every time someone checks in code using the above settings. This build server (and automated testing) support now works with both TFS and Git based source control repositories. Connecting a Team Foundation Services project to Windows Azure Once I have a source repository hosted in Team Foundation Services with Automated Builds and Testing set up, I can then go even further and set it up so that it will be automatically deployed to Windows Azure when a source code commit is made to the repository (assuming the Build + Tests pass). Enabling this is now really easy. To set this up with a Windows Azure Web Site simply use the New->Compute->Web Site->Custom Create command inside the Windows Azure Management Portal. This will create a dialog like below. I gave the web site a name and then made sure the “Publish from source control” checkbox was selected: When we click next we’ll be prompted for the location of the source repository. We’ll select “Team Foundation Services”: Once we do this we’ll be prompted for our Team Foundation Services account that our source repository is hosted under (in this case my TFS account is “scottguthrie”): When we click the “Authorize Now” button we’ll be prompted to give Windows Azure permissions to connect to the Team Foundation Services account. Once we do this we’ll be prompted to pick the source repository we want to connect to. Starting with today’s Windows Azure release you can now connect to both TFS and Git based source repositories. This new support allows me to connect to the “SimpleContinuousDeploymentTest” respository we created earlier: Clicking the finish button will then create the Web Site with the continuous delivery hooks setup with Team Foundation Services. Now every time someone pushes source control to the repository in Team Foundation Services, it will kick off an automated build, run all of the unit tests in the solution , and if they pass the app will be automatically deployed to our Web Site in Windows Azure. You can monitor the history and status of these automated deployments using the Deployments tab within the Web Site: This enables a really slick continuous delivery workflow, and enables you to build and deploy apps in a really nice way. Developer Analytics: New Relic support for Web Sites + Mobile Services With today’s Windows Azure release we are making it really easy to enable Developer Analytics and Monitoring support with both Windows Azure Web Site and Windows Azure Mobile Services. We are partnering with New Relic, who provide a great dev analytics and app performance monitoring offering, to enable this - and we have updated the Windows Azure Management Portal to make it really easy to configure. Enabling New Relic with a Windows Azure Web Site Enabling New Relic support with a Windows Azure Web Site is now really easy. Simply navigate to the Configure tab of a Web Site and scroll down to the “developer analytics” section that is now within it: Clicking the “add-on” button will display some additional UI. If you don’t already have a New Relic subscription, you can click the “view windows azure store” button to obtain a subscription (note: New Relic has a perpetually free tier so you can enable it even without paying anything): Clicking the “view windows azure store” button will launch the integrated Windows Azure Store experience we have within the Windows Azure Management Portal. You can use this to browse from a variety of great add-on services – including New Relic: Select “New Relic” within the dialog above, then click the next button, and you’ll be able to choose which type of New Relic subscription you wish to purchase. For this demo we’ll simply select the “Free Standard Version” – which does not cost anything and can be used forever: Once we’ve signed-up for our New Relic subscription and added it to our Windows Azure account, we can go back to the Web Site’s configuration tab and choose to use the New Relic add-on with our Windows Azure Web Site. We can do this by simply selecting it from the “add-on” dropdown (it is automatically populated within it once we have a New Relic subscription in our account): Clicking the “Save” button will then cause the Windows Azure Management Portal to automatically populate all of the needed New Relic configuration settings to our Web Site: Deploying the New Relic Agent as part of a Web Site The final step to enable developer analytics using New Relic is to add the New Relic runtime agent to our web app. We can do this within Visual Studio by right-clicking on our web project and selecting the “Manage NuGet Packages” context menu: This will bring up the NuGet package manager. You can search for “New Relic” within it to find the New Relic agent. Note that there is both a 32-bit and 64-bit edition of it – make sure to install the version that matches how your Web Site is running within Windows Azure (note: you can configure your Web Site to run in either 32-bit or 64-bit mode using the Web Site’s “Configuration” tab within the Windows Azure Management Portal): Once we install the NuGet package we are all set to go. We’ll simply re-publish the web site again to Windows Azure and New Relic will now automatically start monitoring the application Monitoring a Web Site using New Relic Now that the application has developer analytics support with New Relic enabled, we can launch the New Relic monitoring portal to start monitoring the health of it. We can do this by clicking on the “Add Ons” tab in the left-hand side of the Windows Azure Management Portal. Then select the New Relic add-on we signed-up for within it. The Windows Azure Management Portal will provide some default information about the add-on when we do this. Clicking the “Manage” button in the tray at the bottom will launch a new browser tab and single-sign us into the New Relic monitoring portal associated with our account: When we do this a new browser tab will launch with the New Relic admin tool loaded within it: We can now see insights into how our app is performing – without having to have written a single line of monitoring code. The New Relic service provides a ton of great built-in monitoring features allowing us to quickly see: Performance times (including browser rendering speed) for the overall site and individual pages. You can optionally set alert thresholds to trigger if the speed does not meet a threshold you specify. Information about where in the world your customers are hitting the site from (and how performance varies by region) Details on the latency performance of external services your web apps are using (for example: SQL, Storage, Twitter, etc) Error information including call stack details for exceptions that have occurred at runtime SQL Server profiling information – including which queries executed against your database and what their performance was And a whole bunch more… The cool thing about New Relic is that you don’t need to write monitoring code within your application to get all of the above reports (plus a lot more). The New Relic agent automatically enables the CLR profiler within applications and automatically captures the information necessary to identify these. This makes it super easy to get started and immediately have a rich developer analytics view for your solutions with very little effort. If you haven’t tried New Relic out yet with Windows Azure I recommend you do so – I think you’ll find it helps you build even better cloud applications. Following the above steps will help you get started and deliver you a really good application monitoring solution in only minutes. Service Bus: Support for partitioned queues and topics With today’s release, we are enabling support within Service Bus for partitioned queues and topics. Enabling partitioning enables you to achieve a higher message throughput and better availability from your queues and topics. Higher message throughput is achieved by implementing multiple message brokers for each partitioned queue and topic. The multiple messaging stores will also provide higher availability. You can create a partitioned queue or topic by simply checking the Enable Partitioning option in the custom create wizard for a Queue or Topic: Read this article to learn more about partitioned queues and topics and how to take advantage of them today. Billing: New Billing Alert Service Today’s Windows Azure update enables a new Billing Alert Service Preview that enables you to get proactive email notifications when your Windows Azure bill goes above a certain monetary threshold that you configure. This makes it easier to manage your bill and avoid potential surprises at the end of the month. With the Billing Alert Service Preview, you can now create email alerts to monitor and manage your monetary credits or your current bill total. To set up an alert first sign-up for the free Billing Alert Service Preview. Then visit the account management page, click on a subscription you have setup, and then navigate to the new Alerts tab that is available: The alerts tab allows you to setup email alerts that will be sent automatically once a certain threshold is hit. For example, by clicking the “add alert” button above I can setup a rule to send myself email anytime my Windows Azure bill goes above $100 for the month: The Billing Alert Service will evolve to support additional aspects of your bill as well as support multiple forms of alerts such as SMS. Try out the new Billing Alert Service Preview today and give us feedback. Summary Today’s Windows Azure release enables a ton of great new scenarios, and makes building applications hosted in the cloud even easier. If you don’t already have a Windows Azure account, you can sign-up for a free trial and start using all of the above features today. Then visit the Windows Azure Developer Center to learn more about how to build apps with it. Hope this helps, Scott P.S. In addition to blogging, I am also now using Twitter for quick updates and to share links. Follow me at: twitter.com/scottgu

Read the article

Problem rendering VBO

- by Onno

I'm developing a game engine using OpenTK. I'm trying to get to grips with the use of VBO's. I've run into some trouble because somehow it doesn't render correctly. Thus far I've used immediate mode to render a test object, a test cube with a texture. namespace SharpEngine.Utility.Mesh { using System; using System.Collections.Generic; using OpenTK; using OpenTK.Graphics; using OpenTK.Graphics.OpenGL; using SharpEngine.Utility; using System.Drawing; public class ImmediateFaceBasedCube : IMesh { private IList<Face> faces = new List<Face>(); public ImmediateFaceBasedCube() { IList<Vector3> allVertices = new List<Vector3>(); //rechtsbovenvoor allVertices.Add(new Vector3(1.0f, 1.0f, 1.0f)); //0 //rechtsbovenachter allVertices.Add(new Vector3(1.0f, 1.0f, -1.0f)); //1 //linksbovenachter allVertices.Add(new Vector3(-1.0f, 1.0f, -1.0f)); //2 //linksbovenvoor allVertices.Add(new Vector3(-1.0f, 1.0f, 1.0f)); //3 //rechtsondervoor allVertices.Add(new Vector3(1.0f, -1.0f, 1.0f)); //4 //rechtsonderachter allVertices.Add(new Vector3(1.0f, -1.0f, -1.0f)); //5 //linksonderachter allVertices.Add(new Vector3(-1.0f, -1.0f, -1.0f)); //6 //linksondervoor allVertices.Add(new Vector3(-1.0f, -1.0f, 1.0f)); //7 IList<Vector2> textureCoordinates = new List<Vector2>(); textureCoordinates.Add(new Vector2(0, 0)); //AA - 0 textureCoordinates.Add(new Vector2(0, 0.3333333f)); //AB - 1 textureCoordinates.Add(new Vector2(0, 0.6666666f)); //AC - 2 textureCoordinates.Add(new Vector2(0, 1)); //AD - 3 textureCoordinates.Add(new Vector2(0.3333333f, 0)); //BA - 4 textureCoordinates.Add(new Vector2(0.3333333f, 0.3333333f)); //BB - 5 textureCoordinates.Add(new Vector2(0.3333333f, 0.6666666f)); //BC - 6 textureCoordinates.Add(new Vector2(0.3333333f, 1)); //BD - 7 textureCoordinates.Add(new Vector2(0.6666666f, 0)); //CA - 8 textureCoordinates.Add(new Vector2(0.6666666f, 0.3333333f)); //CB - 9 textureCoordinates.Add(new Vector2(0.6666666f, 0.6666666f)); //CC -10 textureCoordinates.Add(new Vector2(0.6666666f, 1)); //CD -11 textureCoordinates.Add(new Vector2(1, 0)); //DA -12 textureCoordinates.Add(new Vector2(1, 0.3333333f)); //DB -13 textureCoordinates.Add(new Vector2(1, 0.6666666f)); //DC -14 textureCoordinates.Add(new Vector2(1, 1)); //DD -15 Vector3 copy1 = new Vector3(-2.0f, -2.5f, -3.5f); IList<Vector3> normals = new List<Vector3>(); normals.Add(new Vector3(0, 1.0f, 0)); //0 normals.Add(new Vector3(0, 0, 1.0f)); //1 normals.Add(new Vector3(1.0f, 0, 0)); //2 normals.Add(new Vector3(0, 0, -1.0f)); //3 normals.Add(new Vector3(-1.0f, 0, 0)); //4 normals.Add(new Vector3(0, -1.0f, 0)); //5 //todo: move vertex normal and texture data to datastructure //todo: VBO based rendering //top face //1 IList<VertexData> verticesT1 = new List<VertexData>(); VertexData T1a = new VertexData(); T1a.Normal = normals[0]; T1a.TexCoord = textureCoordinates[5]; T1a.Position = allVertices[3]; verticesT1.Add(T1a); VertexData T1b = new VertexData(); T1b.Normal = normals[0]; T1b.TexCoord = textureCoordinates[9]; T1b.Position = allVertices[0]; verticesT1.Add(T1b); VertexData T1c = new VertexData(); T1c.Normal = normals[0]; T1c.TexCoord = textureCoordinates[10]; T1c.Position = allVertices[1]; verticesT1.Add(T1c); Face F1 = new Face(verticesT1); faces.Add(F1); //2 IList<VertexData> verticesT2 = new List<VertexData>(); VertexData T2a = new VertexData(); T2a.Normal = normals[0]; T2a.TexCoord = textureCoordinates[10]; T2a.Position = allVertices[1]; verticesT2.Add(T2a); VertexData T2b = new VertexData(); T2b.Normal = normals[0]; T2b.TexCoord = textureCoordinates[6]; T2b.Position = allVertices[2]; verticesT2.Add(T2b); VertexData T2c = new VertexData(); T2c.Normal = normals[0]; T2c.TexCoord = textureCoordinates[5]; T2c.Position = allVertices[3]; verticesT2.Add(T2c); Face F2 = new Face(verticesT2); faces.Add(F2); //front face //3 IList<VertexData> verticesT3 = new List<VertexData>(); VertexData T3a = new VertexData(); T3a.Normal = normals[1]; T3a.TexCoord = textureCoordinates[1]; T3a.Position = allVertices[3]; verticesT3.Add(T3a); VertexData T3b = new VertexData(); T3b.Normal = normals[1]; T3b.TexCoord = textureCoordinates[0]; T3b.Position = allVertices[7]; verticesT3.Add(T3b); VertexData T3c = new VertexData(); T3c.Normal = normals[1]; T3c.TexCoord = textureCoordinates[5]; T3c.Position = allVertices[0]; verticesT3.Add(T3c); Face F3 = new Face(verticesT3); faces.Add(F3); //4 IList<VertexData> verticesT4 = new List<VertexData>(); VertexData T4a = new VertexData(); T4a.Normal = normals[1]; T4a.TexCoord = textureCoordinates[5]; T4a.Position = allVertices[0]; verticesT4.Add(T4a); VertexData T4b = new VertexData(); T4b.Normal = normals[1]; T4b.TexCoord = textureCoordinates[0]; T4b.Position = allVertices[7]; verticesT4.Add(T4b); VertexData T4c = new VertexData(); T4c.Normal = normals[1]; T4c.TexCoord = textureCoordinates[4]; T4c.Position = allVertices[4]; verticesT4.Add(T4c); Face F4 = new Face(verticesT4); faces.Add(F4); //right face //5 IList<VertexData> verticesT5 = new List<VertexData>(); VertexData T5a = new VertexData(); T5a.Normal = normals[2]; T5a.TexCoord = textureCoordinates[2]; T5a.Position = allVertices[0]; verticesT5.Add(T5a); VertexData T5b = new VertexData(); T5b.Normal = normals[2]; T5b.TexCoord = textureCoordinates[1]; T5b.Position = allVertices[4]; verticesT5.Add(T5b); VertexData T5c = new VertexData(); T5c.Normal = normals[2]; T5c.TexCoord = textureCoordinates[6]; T5c.Position = allVertices[1]; verticesT5.Add(T5c); Face F5 = new Face(verticesT5); faces.Add(F5); //6 IList<VertexData> verticesT6 = new List<VertexData>(); VertexData T6a = new VertexData(); T6a.Normal = normals[2]; T6a.TexCoord = textureCoordinates[1]; T6a.Position = allVertices[4]; verticesT6.Add(T6a); VertexData T6b = new VertexData(); T6b.Normal = normals[2]; T6b.TexCoord = textureCoordinates[5]; T6b.Position = allVertices[5]; verticesT6.Add(T6b); VertexData T6c = new VertexData(); T6c.Normal = normals[2]; T6c.TexCoord = textureCoordinates[6]; T6c.Position = allVertices[1]; verticesT6.Add(T6c); Face F6 = new Face(verticesT6); faces.Add(F6); //back face //7 IList<VertexData> verticesT7 = new List<VertexData>(); VertexData T7a = new VertexData(); T7a.Normal = normals[3]; T7a.TexCoord = textureCoordinates[4]; T7a.Position = allVertices[5]; verticesT7.Add(T7a); VertexData T7b = new VertexData(); T7b.Normal = normals[3]; T7b.TexCoord = textureCoordinates[9]; T7b.Position = allVertices[2]; verticesT7.Add(T7b); VertexData T7c = new VertexData(); T7c.Normal = normals[3]; T7c.TexCoord = textureCoordinates[5]; T7c.Position = allVertices[1]; verticesT7.Add(T7c); Face F7 = new Face(verticesT7); faces.Add(F7); //8 IList<VertexData> verticesT8 = new List<VertexData>(); VertexData T8a = new VertexData(); T8a.Normal = normals[3]; T8a.TexCoord = textureCoordinates[9]; T8a.Position = allVertices[2]; verticesT8.Add(T8a); VertexData T8b = new VertexData(); T8b.Normal = normals[3]; T8b.TexCoord = textureCoordinates[4]; T8b.Position = allVertices[5]; verticesT8.Add(T8b); VertexData T8c = new VertexData(); T8c.Normal = normals[3]; T8c.TexCoord = textureCoordinates[8]; T8c.Position = allVertices[6]; verticesT8.Add(T8c); Face F8 = new Face(verticesT8); faces.Add(F8); //left face //9 IList<VertexData> verticesT9 = new List<VertexData>(); VertexData T9a = new VertexData(); T9a.Normal = normals[4]; T9a.TexCoord = textureCoordinates[8]; T9a.Position = allVertices[6]; verticesT9.Add(T9a); VertexData T9b = new VertexData(); T9b.Normal = normals[4]; T9b.TexCoord = textureCoordinates[13]; T9b.Position = allVertices[3]; verticesT9.Add(T9b); VertexData T9c = new VertexData(); T9c.Normal = normals[4]; T9c.TexCoord = textureCoordinates[9]; T9c.Position = allVertices[2]; verticesT9.Add(T9c); Face F9 = new Face(verticesT9); faces.Add(F9); //10 IList<VertexData> verticesT10 = new List<VertexData>(); VertexData T10a = new VertexData(); T10a.Normal = normals[4]; T10a.TexCoord = textureCoordinates[8]; T10a.Position = allVertices[6]; verticesT10.Add(T10a); VertexData T10b = new VertexData(); T10b.Normal = normals[4]; T10b.TexCoord = textureCoordinates[12]; T10b.Position = allVertices[7]; verticesT10.Add(T10b); VertexData T10c = new VertexData(); T10c.Normal = normals[4]; T10c.TexCoord = textureCoordinates[13]; T10c.Position = allVertices[3]; verticesT10.Add(T10c); Face F10 = new Face(verticesT10); faces.Add(F10); //bottom face //11 IList<VertexData> verticesT11 = new List<VertexData>(); VertexData T11a = new VertexData(); T11a.Normal = normals[5]; T11a.TexCoord = textureCoordinates[10]; T11a.Position = allVertices[7]; verticesT11.Add(T11a); VertexData T11b = new VertexData(); T11b.Normal = normals[5]; T11b.TexCoord = textureCoordinates[9]; T11b.Position = allVertices[6]; verticesT11.Add(T11b); VertexData T11c = new VertexData(); T11c.Normal = normals[5]; T11c.TexCoord = textureCoordinates[14]; T11c.Position = allVertices[4]; verticesT11.Add(T11c); Face F11 = new Face(verticesT11); faces.Add(F11); //12 IList<VertexData> verticesT12 = new List<VertexData>(); VertexData T12a = new VertexData(); T12a.Normal = normals[5]; T12a.TexCoord = textureCoordinates[13]; T12a.Position = allVertices[5]; verticesT12.Add(T12a); VertexData T12b = new VertexData(); T12b.Normal = normals[5]; T12b.TexCoord = textureCoordinates[14]; T12b.Position = allVertices[4]; verticesT12.Add(T12b); VertexData T12c = new VertexData(); T12c.Normal = normals[5]; T12c.TexCoord = textureCoordinates[9]; T12c.Position = allVertices[6]; verticesT12.Add(T12c); Face F12 = new Face(verticesT12); faces.Add(F12); } public void draw() { GL.Begin(BeginMode.Triangles); foreach (Face face in faces) { foreach (VertexData datapoint in face.verticesWithTexCoords) { GL.Normal3(datapoint.Normal); GL.TexCoord2(datapoint.TexCoord); GL.Vertex3(datapoint.Position); } } GL.End(); } } } Gets me this very nice picture: The immediate mode cube renders nicely and taught me a bit on how to use OpenGL, but VBO's are the way to go. Since I read on the OpenTK forums that OpenTK has problems doing VA's or DL's, I decided to skip using those. Now, I've tried to change this cube to a VBO by using the same vertex, normal and tc collections, and making float arrays from them by using the coordinates in combination with uint arrays which contain the index numbers from the immediate cube. (see the private functions at end of the code sample) Somehow this only renders two triangles namespace SharpEngine.Utility.Mesh { using System; using System.Collections.Generic; using OpenTK; using OpenTK.Graphics; using OpenTK.Graphics.OpenGL; using SharpEngine.Utility; using System.Drawing; public class VBOFaceBasedCube : IMesh { private int VerticesVBOID; private int VerticesVBOStride; private int VertexCount; private int ELementBufferObjectID; private int textureCoordinateVBOID; private int textureCoordinateVBOStride; //private int textureCoordinateArraySize; private int normalVBOID; private int normalVBOStride; public VBOFaceBasedCube() { IList<Vector3> allVertices = new List<Vector3>(); //rechtsbovenvoor allVertices.Add(new Vector3(1.0f, 1.0f, 1.0f)); //0 //rechtsbovenachter allVertices.Add(new Vector3(1.0f, 1.0f, -1.0f)); //1 //linksbovenachter allVertices.Add(new Vector3(-1.0f, 1.0f, -1.0f)); //2 //linksbovenvoor allVertices.Add(new Vector3(-1.0f, 1.0f, 1.0f)); //3 //rechtsondervoor allVertices.Add(new Vector3(1.0f, -1.0f, 1.0f)); //4 //rechtsonderachter allVertices.Add(new Vector3(1.0f, -1.0f, -1.0f)); //5 //linksonderachter allVertices.Add(new Vector3(-1.0f, -1.0f, -1.0f)); //6 //linksondervoor allVertices.Add(new Vector3(-1.0f, -1.0f, 1.0f)); //7 IList<Vector2> textureCoordinates = new List<Vector2>(); textureCoordinates.Add(new Vector2(0, 0)); //AA - 0 textureCoordinates.Add(new Vector2(0, 0.3333333f)); //AB - 1 textureCoordinates.Add(new Vector2(0, 0.6666666f)); //AC - 2 textureCoordinates.Add(new Vector2(0, 1)); //AD - 3 textureCoordinates.Add(new Vector2(0.3333333f, 0)); //BA - 4 textureCoordinates.Add(new Vector2(0.3333333f, 0.3333333f)); //BB - 5 textureCoordinates.Add(new Vector2(0.3333333f, 0.6666666f)); //BC - 6 textureCoordinates.Add(new Vector2(0.3333333f, 1)); //BD - 7 textureCoordinates.Add(new Vector2(0.6666666f, 0)); //CA - 8 textureCoordinates.Add(new Vector2(0.6666666f, 0.3333333f)); //CB - 9 textureCoordinates.Add(new Vector2(0.6666666f, 0.6666666f)); //CC -10 textureCoordinates.Add(new Vector2(0.6666666f, 1)); //CD -11 textureCoordinates.Add(new Vector2(1, 0)); //DA -12 textureCoordinates.Add(new Vector2(1, 0.3333333f)); //DB -13 textureCoordinates.Add(new Vector2(1, 0.6666666f)); //DC -14 textureCoordinates.Add(new Vector2(1, 1)); //DD -15 Vector3 copy1 = new Vector3(-2.0f, -2.5f, -3.5f); IList<Vector3> normals = new List<Vector3>(); normals.Add(new Vector3(0, 1.0f, 0)); //0 normals.Add(new Vector3(0, 0, 1.0f)); //1 normals.Add(new Vector3(1.0f, 0, 0)); //2 normals.Add(new Vector3(0, 0, -1.0f)); //3 normals.Add(new Vector3(-1.0f, 0, 0)); //4 normals.Add(new Vector3(0, -1.0f, 0)); //5 //todo: VBO based rendering uint[] vertexElements = { 3,0,1, //01 1,2,3, //02 3,7,0, //03 0,7,4, //04 0,4,1, //05 4,5,1, //06 5,2,1, //07 2,5,6, //08 6,3,2, //09 6,7,5, //10 7,6,4, //11 5,4,6 //12 }; VertexCount = vertexElements.Length; IList<uint> vertexElementList = new List<uint>(vertexElements); uint[] normalElements = { 0,0,0, 0,0,0, 1,1,1, 1,1,1, 2,2,2, 2,2,2, 3,3,3, 3,3,3, 4,4,4, 4,4,4, 5,5,5, 5,5,5 }; IList<uint> normalElementList = new List<uint>(normalElements); uint[] textureIndexArray = { 5,9,10, 10,6,5, 1,0,5, 5,0,4, 2,1,6, 1,5,6, 4,9,5, 9,4,8, 8,13,9, 8,12,13, 10,9,14, 13,14,9 }; //textureCoordinateArraySize = textureIndexArray.Length; IList<uint> textureIndexList = new List<uint>(textureIndexArray); LoadVBO(allVertices, normals, textureCoordinates, vertexElements, normalElementList, textureIndexList); } public void draw() { //bind vertices //bind elements //bind normals //bind texture coordinates GL.EnableClientState(ArrayCap.VertexArray); GL.EnableClientState(ArrayCap.NormalArray); GL.EnableClientState(ArrayCap.TextureCoordArray); GL.BindBuffer(BufferTarget.ArrayBuffer, VerticesVBOID); GL.VertexPointer(3, VertexPointerType.Float, VerticesVBOStride, 0); GL.BindBuffer(BufferTarget.ArrayBuffer, normalVBOID); GL.NormalPointer(NormalPointerType.Float, normalVBOStride, 0); GL.BindBuffer(BufferTarget.ArrayBuffer, textureCoordinateVBOID); GL.TexCoordPointer(2, TexCoordPointerType.Float, textureCoordinateVBOStride, 0); GL.BindBuffer(BufferTarget.ElementArrayBuffer, ELementBufferObjectID); GL.DrawElements(BeginMode.Polygon, VertexCount, DrawElementsType.UnsignedShort, 0); } //loads a static VBO void LoadVBO(IList<Vector3> vertices, IList<Vector3> normals, IList<Vector2> texcoords, uint[] elements, IList<uint> normalIndices, IList<uint> texCoordIndices) { int size; //todo // To create a VBO: // 1) Generate the buffer handles for the vertex and element buffers. // 2) Bind the vertex buffer handle and upload your vertex data. Check that the buffer was uploaded correctly. // 3) Bind the element buffer handle and upload your element data. Check that the buffer was uploaded correctly. float[] verticesArray = convertVector3fListToFloatArray(vertices); float[] normalsArray = createFloatArrayFromListOfVector3ElementsAndIndices(normals, normalIndices); float[] textureCoordinateArray = createFloatArrayFromListOfVector2ElementsAndIndices(texcoords, texCoordIndices); GL.GenBuffers(1, out VerticesVBOID); GL.BindBuffer(BufferTarget.ArrayBuffer, VerticesVBOID); Console.WriteLine("load 1 - vertices"); VerticesVBOStride = BlittableValueType.StrideOf(verticesArray); GL.BufferData(BufferTarget.ArrayBuffer, (IntPtr)(verticesArray.Length * sizeof(float)), verticesArray, BufferUsageHint.StaticDraw); GL.GetBufferParameter(BufferTarget.ArrayBuffer, BufferParameterName.BufferSize, out size); if (verticesArray.Length * BlittableValueType.StrideOf(verticesArray) != size) { throw new ApplicationException("Vertex data not uploaded correctly"); } else { Console.WriteLine("load 1 finished ok"); size = 0; } Console.WriteLine("load 2 - elements"); GL.GenBuffers(1, out ELementBufferObjectID); GL.BindBuffer(BufferTarget.ElementArrayBuffer, ELementBufferObjectID); GL.BufferData(BufferTarget.ElementArrayBuffer, (IntPtr)(elements.Length * sizeof(uint)), elements, BufferUsageHint.StaticDraw); GL.GetBufferParameter(BufferTarget.ElementArrayBuffer, BufferParameterName.BufferSize, out size); if (elements.Length * sizeof(uint) != size) { throw new ApplicationException("Element data not uploaded correctly"); } else { size = 0; Console.WriteLine("load 2 finished ok"); } GL.GenBuffers(1, out normalVBOID); GL.BindBuffer(BufferTarget.ArrayBuffer, normalVBOID); Console.WriteLine("load 3 - normals"); normalVBOStride = BlittableValueType.StrideOf(normalsArray); GL.BufferData(BufferTarget.ArrayBuffer, (IntPtr)(normalsArray.Length * sizeof(float)), normalsArray, BufferUsageHint.StaticDraw); GL.GetBufferParameter(BufferTarget.ArrayBuffer, BufferParameterName.BufferSize, out size); Console.WriteLine("load 3 - pre check"); if (normalsArray.Length * BlittableValueType.StrideOf(normalsArray) != size) { throw new ApplicationException("Normal data not uploaded correctly"); } else { Console.WriteLine("load 3 finished ok"); size = 0; } GL.GenBuffers(1, out textureCoordinateVBOID); GL.BindBuffer(BufferTarget.ArrayBuffer, textureCoordinateVBOID); Console.WriteLine("load 4- texture coordinates"); textureCoordinateVBOStride = BlittableValueType.StrideOf(textureCoordinateArray); GL.BufferData(BufferTarget.ArrayBuffer, (IntPtr)(textureCoordinateArray.Length * textureCoordinateVBOStride), textureCoordinateArray, BufferUsageHint.StaticDraw); GL.GetBufferParameter(BufferTarget.ArrayBuffer, BufferParameterName.BufferSize, out size); if (textureCoordinateArray.Length * BlittableValueType.StrideOf(textureCoordinateArray) != size) { throw new ApplicationException("texture coordinate data not uploaded correctly"); } else { Console.WriteLine("load 3 finished ok"); size = 0; } } //used to convert vertex arrayss for use with VBO's private float[] convertVector3fListToFloatArray(IList<Vector3> input) { int arrayElementCount = input.Count * 3; float[] output = new float[arrayElementCount]; int fillCount = 0; foreach (Vector3 v in input) { output[fillCount] = v.X; output[fillCount + 1] = v.Y; output[fillCount + 2] = v.Z; fillCount += 3; } return output; } //used for converting texture coordinate arrays for use with VBO's private float[] convertVector2List_to_floatArray(IList<Vector2> input) { int arrayElementCount = input.Count * 2; float[] output = new float[arrayElementCount]; int fillCount = 0; foreach (Vector2 v in input) { output[fillCount] = v.X; output[fillCount + 1] = v.Y; fillCount += 2; } return output; } //used to create an array of floats from private float[] createFloatArrayFromListOfVector3ElementsAndIndices(IList<Vector3> inputVectors, IList<uint> indices) { int arrayElementCount = inputVectors.Count * indices.Count * 3; float[] output = new float[arrayElementCount]; int fillCount = 0; foreach (int i in indices) { output[fillCount] = inputVectors[i].X; output[fillCount + 1] = inputVectors[i].Y; output[fillCount + 2] = inputVectors[i].Z; fillCount += 3; } return output; } private float[] createFloatArrayFromListOfVector2ElementsAndIndices(IList<Vector2> inputVectors, IList<uint> indices) { int arrayElementCount = inputVectors.Count * indices.Count * 2; float[] output = new float[arrayElementCount]; int fillCount = 0; foreach (int i in indices) { output[fillCount] = inputVectors[i].X; output[fillCount + 1] = inputVectors[i].Y; fillCount += 2; } return output; } } } This code will only render two triangles and they're nothing like I had in mind: I've done some searching. In some other questions I read that, if I did something wrong, I'd get no rendering at all. Clearly, something gets sent to the GFX card, but it might be that I'm not sending the right data. I've tried altering the sequence in which the triangles are rendered by swapping some of the index numbers in the vert, tc and normal index arrays, but this doesn't seem to be of any effect. I'm slightly lost here. What am I doing wrong here?

Read the article

Using R to Analyze G1GC Log Files

- by user12620111

Using R to Analyze G1GC Log Files body, td { font-family: sans-serif; background-color: white; font-size: 12px; margin: 8px; } tt, code, pre { font-family: 'DejaVu Sans Mono', 'Droid Sans Mono', 'Lucida Console', Consolas, Monaco, monospace; } h1 { font-size:2.2em; } h2 { font-size:1.8em; } h3 { font-size:1.4em; } h4 { font-size:1.0em; } h5 { font-size:0.9em; } h6 { font-size:0.8em; } a:visited { color: rgb(50%, 0%, 50%); } pre { margin-top: 0; max-width: 95%; border: 1px solid #ccc; white-space: pre-wrap; } pre code { display: block; padding: 0.5em; } code.r, code.cpp { background-color: #F8F8F8; } table, td, th { border: none; } blockquote { color:#666666; margin:0; padding-left: 1em; border-left: 0.5em #EEE solid; } hr { height: 0px; border-bottom: none; border-top-width: thin; border-top-style: dotted; border-top-color: #999999; } @media print { * { background: transparent !important; color: black !important; filter:none !important; -ms-filter: none !important; } body { font-size:12pt; max-width:100%; } a, a:visited { text-decoration: underline; } hr { visibility: hidden; page-break-before: always; } pre, blockquote { padding-right: 1em; page-break-inside: avoid; } tr, img { page-break-inside: avoid; } img { max-width: 100% !important; } @page :left { margin: 15mm 20mm 15mm 10mm; } @page :right { margin: 15mm 10mm 15mm 20mm; } p, h2, h3 { orphans: 3; widows: 3; } h2, h3 { page-break-after: avoid; } } pre .operator, pre .paren { color: rgb(104, 118, 135) } pre .literal { color: rgb(88, 72, 246) } pre .number { color: rgb(0, 0, 205); } pre .comment { color: rgb(76, 136, 107); } pre .keyword { color: rgb(0, 0, 255); } pre .identifier { color: rgb(0, 0, 0); } pre .string { color: rgb(3, 106, 7); } var hljs=new function(){function m(p){return p.replace(/&/gm,"&").replace(/"}while(y.length||w.length){var v=u().splice(0,1)[0];z+=m(x.substr(q,v.offset-q));q=v.offset;if(v.event=="start"){z+=t(v.node);s.push(v.node)}else{if(v.event=="stop"){var p,r=s.length;do{r--;p=s[r];z+=("")}while(p!=v.node);s.splice(r,1);while(r'+M[0]+""}else{r+=M[0]}O=P.lR.lastIndex;M=P.lR.exec(L)}return r+L.substr(O,L.length-O)}function J(L,M){if(M.sL&&e[M.sL]){var r=d(M.sL,L);x+=r.keyword_count;return r.value}else{return F(L,M)}}function I(M,r){var L=M.cN?'':"";if(M.rB){y+=L;M.buffer=""}else{if(M.eB){y+=m(r)+L;M.buffer=""}else{y+=L;M.buffer=r}}D.push(M);A+=M.r}function G(N,M,Q){var R=D[D.length-1];if(Q){y+=J(R.buffer+N,R);return false}var P=q(M,R);if(P){y+=J(R.buffer+N,R);I(P,M);return P.rB}var L=v(D.length-1,M);if(L){var O=R.cN?"":"";if(R.rE){y+=J(R.buffer+N,R)+O}else{if(R.eE){y+=J(R.buffer+N,R)+O+m(M)}else{y+=J(R.buffer+N+M,R)+O}}while(L1){O=D[D.length-2].cN?"":"";y+=O;L--;D.length--}var r=D[D.length-1];D.length--;D[D.length-1].buffer="";if(r.starts){I(r.starts,"")}return R.rE}if(w(M,R)){throw"Illegal"}}var E=e[B];var D=[E.dM];var A=0;var x=0;var y="";try{var s,u=0;E.dM.buffer="";do{s=p(C,u);var t=G(s[0],s[1],s[2]);u+=s[0].length;if(!t){u+=s[1].length}}while(!s[2]);if(D.length1){throw"Illegal"}return{r:A,keyword_count:x,value:y}}catch(H){if(H=="Illegal"){return{r:0,keyword_count:0,value:m(C)}}else{throw H}}}function g(t){var p={keyword_count:0,r:0,value:m(t)};var r=p;for(var q in e){if(!e.hasOwnProperty(q)){continue}var s=d(q,t);s.language=q;if(s.keyword_count+s.rr.keyword_count+r.r){r=s}if(s.keyword_count+s.rp.keyword_count+p.r){r=p;p=s}}if(r.language){p.second_best=r}return p}function i(r,q,p){if(q){r=r.replace(/^((]+|\t)+)/gm,function(t,w,v,u){return w.replace(/\t/g,q)})}if(p){r=r.replace(/\n/g,"")}return r}function n(t,w,r){var x=h(t,r);var v=a(t);var y,s;if(v){y=d(v,x)}else{return}var q=c(t);if(q.length){s=document.createElement("pre");s.innerHTML=y.value;y.value=k(q,c(s),x)}y.value=i(y.value,w,r);var u=t.className;if(!u.match("(\\s|^)(language-)?"+v+"(\\s|$)")){u=u?(u+" "+v):v}if(/MSIE [678]/.test(navigator.userAgent)&&t.tagName=="CODE"&&t.parentNode.tagName=="PRE"){s=t.parentNode;var p=document.createElement("div");p.innerHTML=""+y.value+"";t=p.firstChild.firstChild;p.firstChild.cN=s.cN;s.parentNode.replaceChild(p.firstChild,s)}else{t.innerHTML=y.value}t.className=u;t.result={language:v,kw:y.keyword_count,re:y.r};if(y.second_best){t.second_best={language:y.second_best.language,kw:y.second_best.keyword_count,re:y.second_best.r}}}function o(){if(o.called){return}o.called=true;var r=document.getElementsByTagName("pre");for(var p=0;p|=||=||=|\\?|\\[|\\{|\\(|\\^|\\^=|\\||\\|=|\\|\\||~";this.ER="(?![\\s\\S])";this.BE={b:"\\\\.",r:0};this.ASM={cN:"string",b:"'",e:"'",i:"\\n",c:[this.BE],r:0};this.QSM={cN:"string",b:'"',e:'"',i:"\\n",c:[this.BE],r:0};this.CLCM={cN:"comment",b:"//",e:"$"};this.CBLCLM={cN:"comment",b:"/\\*",e:"\\*/"};this.HCM={cN:"comment",b:"#",e:"$"};this.NM={cN:"number",b:this.NR,r:0};this.CNM={cN:"number",b:this.CNR,r:0};this.BNM={cN:"number",b:this.BNR,r:0};this.inherit=function(r,s){var p={};for(var q in r){p[q]=r[q]}if(s){for(var q in s){p[q]=s[q]}}return p}}();hljs.LANGUAGES.cpp=function(){var a={keyword:{"false":1,"int":1,"float":1,"while":1,"private":1,"char":1,"catch":1,"export":1,virtual:1,operator:2,sizeof:2,dynamic_cast:2,typedef:2,const_cast:2,"const":1,struct:1,"for":1,static_cast:2,union:1,namespace:1,unsigned:1,"long":1,"throw":1,"volatile":2,"static":1,"protected":1,bool:1,template:1,mutable:1,"if":1,"public":1,friend:2,"do":1,"return":1,"goto":1,auto:1,"void":2,"enum":1,"else":1,"break":1,"new":1,extern:1,using:1,"true":1,"class":1,asm:1,"case":1,typeid:1,"short":1,reinterpret_cast:2,"default":1,"double":1,register:1,explicit:1,signed:1,typename:1,"try":1,"this":1,"switch":1,"continue":1,wchar_t:1,inline:1,"delete":1,alignof:1,char16_t:1,char32_t:1,constexpr:1,decltype:1,noexcept:1,nullptr:1,static_assert:1,thread_local:1,restrict:1,_Bool:1,complex:1},built_in:{std:1,string:1,cin:1,cout:1,cerr:1,clog:1,stringstream:1,istringstream:1,ostringstream:1,auto_ptr:1,deque:1,list:1,queue:1,stack:1,vector:1,map:1,set:1,bitset:1,multiset:1,multimap:1,unordered_set:1,unordered_map:1,unordered_multiset:1,unordered_multimap:1,array:1,shared_ptr:1}};return{dM:{k:a,i:"",k:a,r:10,c:["self"]}]}}}();hljs.LANGUAGES.r={dM:{c:[hljs.HCM,{cN:"number",b:"\\b0[xX][0-9a-fA-F]+[Li]?\\b",e:hljs.IMMEDIATE_RE,r:0},{cN:"number",b:"\\b\\d+(?:[eE][+\\-]?\\d*)?L\\b",e:hljs.IMMEDIATE_RE,r:0},{cN:"number",b:"\\b\\d+\\.(?!\\d)(?:i\\b)?",e:hljs.IMMEDIATE_RE,r:1},{cN:"number",b:"\\b\\d+(?:\\.\\d*)?(?:[eE][+\\-]?\\d*)?i?\\b",e:hljs.IMMEDIATE_RE,r:0},{cN:"number",b:"\\.\\d+(?:[eE][+\\-]?\\d*)?i?\\b",e:hljs.IMMEDIATE_RE,r:1},{cN:"keyword",b:"(?:tryCatch|library|setGeneric|setGroupGeneric)\\b",e:hljs.IMMEDIATE_RE,r:10},{cN:"keyword",b:"\\.\\.\\.",e:hljs.IMMEDIATE_RE,r:10},{cN:"keyword",b:"\\.\\.\\d+(?![\\w.])",e:hljs.IMMEDIATE_RE,r:10},{cN:"keyword",b:"\\b(?:function)",e:hljs.IMMEDIATE_RE,r:2},{cN:"keyword",b:"(?:if|in|break|next|repeat|else|for|return|switch|while|try|stop|warning|require|attach|detach|source|setMethod|setClass)\\b",e:hljs.IMMEDIATE_RE,r:1},{cN:"literal",b:"(?:NA|NA_integer_|NA_real_|NA_character_|NA_complex_)\\b",e:hljs.IMMEDIATE_RE,r:10},{cN:"literal",b:"(?:NULL|TRUE|FALSE|T|F|Inf|NaN)\\b",e:hljs.IMMEDIATE_RE,r:1},{cN:"identifier",b:"[a-zA-Z.][a-zA-Z0-9._]*\\b",e:hljs.IMMEDIATE_RE,r:0},{cN:"operator",b:"|=|| Using R to Analyze G1GC Log Files Using R to Analyze G1GC Log Files Introduction Working in Oracle Platform Integration gives an engineer opportunities to work on a wide array of technologies. My team’s goal is to make Oracle applications run best on the Solaris/SPARC platform. When looking for bottlenecks in a modern applications, one needs to be aware of not only how the CPUs and operating system are executing, but also network, storage, and in some cases, the Java Virtual Machine. I was recently presented with about 1.5 GB of Java Garbage First Garbage Collector log file data. If you’re not familiar with the subject, you might want to review Garbage First Garbage Collector Tuning by Monica Beckwith. The customer had been running Java HotSpot 1.6.0_31 to host a web application server. I was told that the Solaris/SPARC server was running a Java process launched using a commmand line that included the following flags: -d64 -Xms9g -Xmx9g -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:InitiatingHeapOccupancyPercent=80 -XX:PermSize=256m -XX:MaxPermSize=256m -XX:+PrintGC -XX:+PrintGCTimeStamps -XX:+PrintHeapAtGC -XX:+PrintGCDateStamps -XX:+PrintFlagsFinal -XX:+DisableExplicitGC -XX:+UnlockExperimentalVMOptions -XX:ParallelGCThreads=8 Several sources on the internet indicate that if I were to print out the 1.5 GB of log files, it would require enough paper to fill the bed of a pick up truck. Of course, it would be fruitless to try to scan the log files by hand. Tools will be required to summarize the contents of the log files. Others have encountered large Java garbage collection log files. There are existing tools to analyze the log files: IBM’s GC toolkit The chewiebug GCViewer gchisto HPjmeter Instead of using one of the other tools listed, I decide to parse the log files with standard Unix tools, and analyze the data with R. Data Cleansing The log files arrived in two different formats. I guess that the difference is that one set of log files was generated using a more verbose option, maybe -XX:+PrintHeapAtGC, and the other set of log files was generated without that option. Format 1 In some of the log files, the log files with the less verbose format, a single trace, i.e. the report of a singe garbage collection event, looks like this: {Heap before GC invocations=12280 (full 61): garbage-first heap total 9437184K, used 7499918K [0xfffffffd00000000, 0xffffffff40000000, 0xffffffff40000000) region size 4096K, 1 young (4096K), 0 survivors (0K) compacting perm gen total 262144K, used 144077K [0xffffffff40000000, 0xffffffff50000000, 0xffffffff50000000) the space 262144K, 54% used [0xffffffff40000000, 0xffffffff48cb3758, 0xffffffff48cb3800, 0xffffffff50000000) No shared spaces configured. 2014-05-14T07:24:00.988-0700: 60586.353: [GC pause (young) 7324M->7320M(9216M), 0.1567265 secs] Heap after GC invocations=12281 (full 61): garbage-first heap total 9437184K, used 7496533K [0xfffffffd00000000, 0xffffffff40000000, 0xffffffff40000000) region size 4096K, 0 young (0K), 0 survivors (0K) compacting perm gen total 262144K, used 144077K [0xffffffff40000000, 0xffffffff50000000, 0xffffffff50000000) the space 262144K, 54% used [0xffffffff40000000, 0xffffffff48cb3758, 0xffffffff48cb3800, 0xffffffff50000000) No shared spaces configured. } A simple grep can be used to extract a summary: $ grep "\[ GC pause (young" g1gc.log 2014-05-13T13:24:35.091-0700: 3.109: [GC pause (young) 20M->5029K(9216M), 0.0146328 secs] 2014-05-13T13:24:35.440-0700: 3.459: [GC pause (young) 9125K->6077K(9216M), 0.0086723 secs] 2014-05-13T13:24:37.581-0700: 5.599: [GC pause (young) 25M->8470K(9216M), 0.0203820 secs] 2014-05-13T13:24:42.686-0700: 10.704: [GC pause (young) 44M->15M(9216M), 0.0288848 secs] 2014-05-13T13:24:48.941-0700: 16.958: [GC pause (young) 51M->20M(9216M), 0.0491244 secs] 2014-05-13T13:24:56.049-0700: 24.066: [GC pause (young) 92M->26M(9216M), 0.0525368 secs] 2014-05-13T13:25:34.368-0700: 62.383: [GC pause (young) 602M->68M(9216M), 0.1721173 secs] But that format wasn't easily read into R, so I needed to be a bit more tricky. I used the following Unix command to create a summary file that was easy for R to read. $ echo "SecondsSinceLaunch BeforeSize AfterSize TotalSize RealTime" $ grep "\[GC pause (young" g1gc.log | grep -v mark | sed -e 's/[A-SU-z,]/ /g' -e 's/->/ /' -e 's/: / /g' | more SecondsSinceLaunch BeforeSize AfterSize TotalSize RealTime 2014-05-13T13:24:35.091-0700 3.109 20 5029 9216 0.0146328 2014-05-13T13:24:35.440-0700 3.459 9125 6077 9216 0.0086723 2014-05-13T13:24:37.581-0700 5.599 25 8470 9216 0.0203820 2014-05-13T13:24:42.686-0700 10.704 44 15 9216 0.0288848 2014-05-13T13:24:48.941-0700 16.958 51 20 9216 0.0491244 2014-05-13T13:24:56.049-0700 24.066 92 26 9216 0.0525368 2014-05-13T13:25:34.368-0700 62.383 602 68 9216 0.1721173 Format 2 In some of the log files, the log files with the more verbose format, a single trace, i.e. the report of a singe garbage collection event, was more complicated than Format 1. Here is a text file with an example of a single G1GC trace in the second format. As you can see, it is quite complicated. It is nice that there is so much information available, but the level of detail can be overwhelming. I wrote this awk script (download) to summarize each trace on a single line. #!/usr/bin/env awk -f BEGIN { printf("SecondsSinceLaunch IncrementalCount FullCount UserTime SysTime RealTime BeforeSize AfterSize TotalSize\n") } ###################### # Save count data from lines that are at the start of each G1GC trace. # Each trace starts out like this: # {Heap before GC invocations=14 (full 0): # garbage-first heap total 9437184K, used 325496K [0xfffffffd00000000, 0xffffffff40000000, 0xffffffff40000000) ###################### /{Heap.*full/{ gsub ( "\\)" , "" ); nf=split($0,a,"="); split(a[2],b," "); getline; if ( match($0, "first") ) { G1GC=1; IncrementalCount=b[1]; FullCount=substr( b[3], 1, length(b[3])-1 ); } else { G1GC=0; } } ###################### # Pull out time stamps that are in lines with this format: # 2014-05-12T14:02:06.025-0700: 94.312: [GC pause (young), 0.08870154 secs] ###################### /GC pause/ { DateTime=$1; SecondsSinceLaunch=substr($2, 1, length($2)-1); } ###################### # Heap sizes are in lines that look like this: # [ 4842M->4838M(9216M)] ###################### /\[ .*]$/ { gsub ( "\\[" , "" ); gsub ( "\ \]" , "" ); gsub ( "->" , " " ); gsub ( "\$ " , " " ); gsub ( "\ $" , " " ); split($0,a," "); if ( split(a[1],b,"M") > 1 ) {BeforeSize=b[1]*1024;} if ( split(a[1],b,"K") > 1 ) {BeforeSize=b[1];} if ( split(a[2],b,"M") > 1 ) {AfterSize=b[1]*1024;} if ( split(a[2],b,"K") > 1 ) {AfterSize=b[1];} if ( split(a[3],b,"M") > 1 ) {TotalSize=b[1]*1024;} if ( split(a[3],b,"K") > 1 ) {TotalSize=b[1];} } ###################### # Emit an output line when you find input that looks like this: # [Times: user=1.41 sys=0.08, real=0.24 secs] ###################### /\[Times/ { if (G1GC==1) { gsub ( "," , "" ); split($2,a,"="); UserTime=a[2]; split($3,a,"="); SysTime=a[2]; split($4,a,"="); RealTime=a[2]; print DateTime,SecondsSinceLaunch,IncrementalCount,FullCount,UserTime,SysTime,RealTime,BeforeSize,AfterSize,TotalSize; G1GC=0; } } The resulting summary is about 25X smaller that the original file, but still difficult for a human to digest. SecondsSinceLaunch IncrementalCount FullCount UserTime SysTime RealTime BeforeSize AfterSize TotalSize ... 2014-05-12T18:36:34.669-0700: 3985.744 561 0 0.57 0.06 0.16 1724416 1720320 9437184 2014-05-12T18:36:34.839-0700: 3985.914 562 0 0.51 0.06 0.19 1724416 1720320 9437184 2014-05-12T18:36:35.069-0700: 3986.144 563 0 0.60 0.04 0.27 1724416 1721344 9437184 2014-05-12T18:36:35.354-0700: 3986.429 564 0 0.33 0.04 0.09 1725440 1722368 9437184 2014-05-12T18:36:35.545-0700: 3986.620 565 0 0.58 0.04 0.17 1726464 1722368 9437184 2014-05-12T18:36:35.726-0700: 3986.801 566 0 0.43 0.05 0.12 1726464 1722368 9437184 2014-05-12T18:36:35.856-0700: 3986.930 567 0 0.30 0.04 0.07 1726464 1723392 9437184 2014-05-12T18:36:35.947-0700: 3987.023 568 0 0.61 0.04 0.26 1727488 1723392 9437184 2014-05-12T18:36:36.228-0700: 3987.302 569 0 0.46 0.04 0.16 1731584 1724416 9437184 Reading the Data into R Once the GC log data had been cleansed, either by processing the first format with the shell script, or by processing the second format with the awk script, it was easy to read the data into R. g1gc.df = read.csv("summary.txt", row.names = NULL, stringsAsFactors=FALSE,sep="") str(g1gc.df) ## 'data.frame': 8307 obs. of 10 variables: ## $ row.names : chr "2014-05-12T14:00:32.868-0700:" "2014-05-12T14:00:33.179-0700:" "2014-05-12T14:00:33.677-0700:" "2014-05-12T14:00:35.538-0700:" ... ## $ SecondsSinceLaunch: num 1.16 1.47 1.97 3.83 6.1 ... ## $ IncrementalCount : int 0 1 2 3 4 5 6 7 8 9 ... ## $ FullCount : int 0 0 0 0 0 0 0 0 0 0 ... ## $ UserTime : num 0.11 0.05 0.04 0.21 0.08 0.26 0.31 0.33 0.34 0.56 ... ## $ SysTime : num 0.04 0.01 0.01 0.05 0.01 0.06 0.07 0.06 0.07 0.09 ... ## $ RealTime : num 0.02 0.02 0.01 0.04 0.02 0.04 0.05 0.04 0.04 0.06 ... ## $ BeforeSize : int 8192 5496 5768 22528 24576 43008 34816 53248 55296 93184 ... ## $ AfterSize : int 1400 1672 2557 4907 7072 14336 16384 18432 19456 21504 ... ## $ TotalSize : int 9437184 9437184 9437184 9437184 9437184 9437184 9437184 9437184 9437184 9437184 ... head(g1gc.df) ## row.names SecondsSinceLaunch IncrementalCount ## 1 2014-05-12T14:00:32.868-0700: 1.161 0 ## 2 2014-05-12T14:00:33.179-0700: 1.472 1 ## 3 2014-05-12T14:00:33.677-0700: 1.969 2 ## 4 2014-05-12T14:00:35.538-0700: 3.830 3 ## 5 2014-05-12T14:00:37.811-0700: 6.103 4 ## 6 2014-05-12T14:00:41.428-0700: 9.720 5 ## FullCount UserTime SysTime RealTime BeforeSize AfterSize TotalSize ## 1 0 0.11 0.04 0.02 8192 1400 9437184 ## 2 0 0.05 0.01 0.02 5496 1672 9437184 ## 3 0 0.04 0.01 0.01 5768 2557 9437184 ## 4 0 0.21 0.05 0.04 22528 4907 9437184 ## 5 0 0.08 0.01 0.02 24576 7072 9437184 ## 6 0 0.26 0.06 0.04 43008 14336 9437184 Basic Statistics Once the data has been read into R, simple statistics are very easy to generate. All of the numbers from high school statistics are available via simple commands. For example, generate a summary of every column: summary(g1gc.df) ## row.names SecondsSinceLaunch IncrementalCount FullCount ## Length:8307 Min. : 1 Min. : 0 Min. : 0.0 ## Class :character 1st Qu.: 9977 1st Qu.:2048 1st Qu.: 0.0 ## Mode :character Median :12855 Median :4136 Median : 12.0 ## Mean :12527 Mean :4156 Mean : 31.6 ## 3rd Qu.:15758 3rd Qu.:6262 3rd Qu.: 61.0 ## Max. :55484 Max. :8391 Max. :113.0 ## UserTime SysTime RealTime BeforeSize ## Min. :0.040 Min. :0.0000 Min. : 0.0 Min. : 5476 ## 1st Qu.:0.470 1st Qu.:0.0300 1st Qu.: 0.1 1st Qu.:5137920 ## Median :0.620 Median :0.0300 Median : 0.1 Median :6574080 ## Mean :0.751 Mean :0.0355 Mean : 0.3 Mean :5841855 ## 3rd Qu.:0.920 3rd Qu.:0.0400 3rd Qu.: 0.2 3rd Qu.:7084032 ## Max. :3.370 Max. :1.5600 Max. :488.1 Max. :8696832 ## AfterSize TotalSize ## Min. : 1380 Min. :9437184 ## 1st Qu.:5002752 1st Qu.:9437184 ## Median :6559744 Median :9437184 ## Mean :5785454 Mean :9437184 ## 3rd Qu.:7054336 3rd Qu.:9437184 ## Max. :8482816 Max. :9437184 Q: What is the total amount of User CPU time spent in garbage collection? sum(g1gc.df$UserTime) ## [1] 6236 As you can see, less than two hours of CPU time was spent in garbage collection. Is that too much? To find the percentage of time spent in garbage collection, divide the number above by total_elapsed_time*CPU_count. In this case, there are a lot of CPU’s and it turns out the the overall amount of CPU time spent in garbage collection isn’t a problem when viewed in isolation. When calculating rates, i.e. events per unit time, you need to ask yourself if the rate is homogenous across the time period in the log file. Does the log file include spikes of high activity that should be separately analyzed? Averaging in data from nights and weekends with data from business hours may alias problems. If you have a reason to suspect that the garbage collection rates include peaks and valleys that need independent analysis, see the “Time Series” section, below. Q: How much garbage is collected on each pass? The amount of heap space that is recovered per GC pass is surprisingly low: At least one collection didn’t recover any data. (“Min.=0”) 25% of the passes recovered 3MB or less. (“1st Qu.=3072”) Half of the GC passes recovered 4MB or less. (“Median=4096”) The average amount recovered was 56MB. (“Mean=56390”) 75% of the passes recovered 36MB or less. (“3rd Qu.=36860”) At least one pass recovered 2GB. (“Max.=2121000”) g1gc.df$Delta = g1gc.df$BeforeSize - g1gc.df$AfterSize summary(g1gc.df$Delta) ## Min. 1st Qu. Median Mean 3rd Qu. Max. ## 0 3070 4100 56400 36900 2120000 Q: What is the maximum User CPU time for a single collection? The worst garbage collection (“Max.”) is many standard deviations away from the mean. The data appears to be right skewed. summary(g1gc.df$UserTime) ## Min. 1st Qu. Median Mean 3rd Qu. Max. ## 0.040 0.470 0.620 0.751 0.920 3.370 sd(g1gc.df$UserTime) ## [1] 0.3966 Basic Graphics Once the data is in R, it is trivial to plot the data with formats including dot plots, line charts, bar charts (simple, stacked, grouped), pie charts, boxplots, scatter plots histograms, and kernel density plots. Histogram of User CPU Time per Collection I don't think that this graph requires any explanation. hist(g1gc.df$UserTime, main="User CPU Time per Collection", xlab="Seconds", ylab="Frequency") Box plot to identify outliers When the initial data is viewed with a box plot, you can see the one crazy outlier in the real time per GC. Save this data point for future analysis and drop the outlier so that it’s not throwing off our statistics. Now the box plot shows many outliers, which will be examined later, using times series analysis. Notice that the scale of the x-axis changes drastically once the crazy outlier is removed. par(mfrow=c(2,1)) boxplot(g1gc.df$UserTime,g1gc.df$SysTime,g1gc.df$RealTime, main="Box Plot of Time per GC\n(dominated by a crazy outlier)", names=c("usr","sys","elapsed"), xlab="Seconds per GC", ylab="Time (Seconds)", horizontal = TRUE, outcol="red") crazy.outlier.df=g1gc.df[g1gc.df$RealTime > 400,] g1gc.df=g1gc.df[g1gc.df$RealTime < 400,] boxplot(g1gc.df$UserTime,g1gc.df$SysTime,g1gc.df$RealTime, main="Box Plot of Time per GC\n(crazy outlier excluded)", names=c("usr","sys","elapsed"), xlab="Seconds per GC", ylab="Time (Seconds)", horizontal = TRUE, outcol="red") box(which = "outer", lty = "solid") Here is the crazy outlier for future analysis: crazy.outlier.df ## row.names SecondsSinceLaunch IncrementalCount ## 8233 2014-05-12T23:15:43.903-0700: 20741 8316 ## FullCount UserTime SysTime RealTime BeforeSize AfterSize TotalSize ## 8233 112 0.55 0.42 488.1 8381440 8235008 9437184 ## Delta ## 8233 146432 R Time Series Data To analyze the garbage collection as a time series, I’ll use Z’s Ordered Observations (zoo). “zoo is the creator for an S3 class of indexed totally ordered observations which includes irregular time series.” require(zoo) ## Loading required package: zoo ## ## Attaching package: 'zoo' ## ## The following objects are masked from 'package:base': ## ## as.Date, as.Date.numeric head(g1gc.df[,1]) ## [1] "2014-05-12T14:00:32.868-0700:" "2014-05-12T14:00:33.179-0700:" ## [3] "2014-05-12T14:00:33.677-0700:" "2014-05-12T14:00:35.538-0700:" ## [5] "2014-05-12T14:00:37.811-0700:" "2014-05-12T14:00:41.428-0700:" options("digits.secs"=3) times=as.POSIXct( g1gc.df[,1], format="%Y-%m-%dT%H:%M:%OS%z:") g1gc.z = zoo(g1gc.df[,-c(1)], order.by=times) head(g1gc.z) ## SecondsSinceLaunch IncrementalCount FullCount ## 2014-05-12 17:00:32.868 1.161 0 0 ## 2014-05-12 17:00:33.178 1.472 1 0 ## 2014-05-12 17:00:33.677 1.969 2 0 ## 2014-05-12 17:00:35.538 3.830 3 0 ## 2014-05-12 17:00:37.811 6.103 4 0 ## 2014-05-12 17:00:41.427 9.720 5 0 ## UserTime SysTime RealTime BeforeSize AfterSize ## 2014-05-12 17:00:32.868 0.11 0.04 0.02 8192 1400 ## 2014-05-12 17:00:33.178 0.05 0.01 0.02 5496 1672 ## 2014-05-12 17:00:33.677 0.04 0.01 0.01 5768 2557 ## 2014-05-12 17:00:35.538 0.21 0.05 0.04 22528 4907 ## 2014-05-12 17:00:37.811 0.08 0.01 0.02 24576 7072 ## 2014-05-12 17:00:41.427 0.26 0.06 0.04 43008 14336 ## TotalSize Delta ## 2014-05-12 17:00:32.868 9437184 6792 ## 2014-05-12 17:00:33.178 9437184 3824 ## 2014-05-12 17:00:33.677 9437184 3211 ## 2014-05-12 17:00:35.538 9437184 17621 ## 2014-05-12 17:00:37.811 9437184 17504 ## 2014-05-12 17:00:41.427 9437184 28672 Example of Two Benchmark Runs in One Log File The data in the following graph is from a different log file, not the one of primary interest to this article. I’m including this image because it is an example of idle periods followed by busy periods. It would be uninteresting to average the rate of garbage collection over the entire log file period. More interesting would be the rate of garbage collect in the two busy periods. Are they the same or different? Your production data may be similar, for example, bursts when employees return from lunch and idle times on weekend evenings, etc. Once the data is in an R Time Series, you can analyze isolated time windows. Clipping the Time Series data Flashing back to our test case… Viewing the data as a time series is interesting. You can see that the work intensive time period is between 9:00 PM and 3:00 AM. Lets clip the data to the interesting period: par(mfrow=c(2,1)) plot(g1gc.z$UserTime, type="h", main="User Time per GC\nTime: Complete Log File", xlab="Time of Day", ylab="CPU Seconds per GC", col="#1b9e77") clipped.g1gc.z=window(g1gc.z, start=as.POSIXct("2014-05-12 21:00:00"), end=as.POSIXct("2014-05-13 03:00:00")) plot(clipped.g1gc.z$UserTime, type="h", main="User Time per GC\nTime: Limited to Benchmark Execution", xlab="Time of Day", ylab="CPU Seconds per GC", col="#1b9e77") box(which = "outer", lty = "solid") Cumulative Incremental and Full GC count Here is the cumulative incremental and full GC count. When the line is very steep, it indicates that the GCs are repeating very quickly. Notice that the scale on the Y axis is different for full vs. incremental. plot(clipped.g1gc.z[,c(2:3)], main="Cumulative Incremental and Full GC count", xlab="Time of Day", col="#1b9e77") GC Analysis of Benchmark Execution using Time Series data In the following series of 3 graphs: The “After Size” show the amount of heap space in use after each garbage collection. Many Java objects are still referenced, i.e. alive, during each garbage collection. This may indicate that the application has a memory leak, or may indicate that the application has a very large memory footprint. Typically, an application's memory footprint plateau's in the early stage of execution. One would expect this graph to have a flat top. The steep decline in the heap space may indicate that the application crashed after 2:00. The second graph shows that the outliers in real execution time, discussed above, occur near 2:00. when the Java heap seems to be quite full. The third graph shows that Full GCs are infrequent during the first few hours of execution. The rate of Full GC's, (the slope of the cummulative Full GC line), changes near midnight. plot(clipped.g1gc.z[,c("AfterSize","RealTime","FullCount")], xlab="Time of Day", col=c("#1b9e77","red","#1b9e77")) GC Analysis of heap recovered Each GC trace includes the amount of heap space in use before and after the individual GC event. During garbage coolection, unreferenced objects are identified, the space holding the unreferenced objects is freed, and thus, the difference in before and after usage indicates how much space has been freed. The following box plot and bar chart both demonstrate the same point - the amount of heap space freed per garbage colloection is surprisingly low. par(mfrow=c(2,1)) boxplot(as.vector(clipped.g1gc.z$Delta), main="Amount of Heap Recovered per GC Pass", xlab="Size in KB", horizontal = TRUE, col="red") hist(as.vector(clipped.g1gc.z$Delta), main="Amount of Heap Recovered per GC Pass", xlab="Size in KB", breaks=100, col="red") box(which = "outer", lty = "solid") This graph is the most interesting. The dark blue area shows how much heap is occupied by referenced Java objects. This represents memory that holds live data. The red fringe at the top shows how much data was recovered after each garbage collection. barplot(clipped.g1gc.z[,c("AfterSize","Delta")], col=c("#7570b3","#e7298a"), xlab="Time of Day", border=NA) legend("topleft", c("Live Objects","Heap Recovered on GC"), fill=c("#7570b3","#e7298a")) box(which = "outer", lty = "solid") When I discuss the data in the log files with the customer, I will ask for an explaination for the large amount of referenced data resident in the Java heap. There are two are posibilities: There is a memory leak and the amount of space required to hold referenced objects will continue to grow, limited only by the maximum heap size. After the maximum heap size is reached, the JVM will throw an “Out of Memory” exception every time that the application tries to allocate a new object. If this is the case, the aplication needs to be debugged to identify why old objects are referenced when they are no longer needed. The application has a legitimate requirement to keep a large amount of data in memory. The customer may want to further increase the maximum heap size. Another possible solution would be to partition the application across multiple cluster nodes, where each node has responsibility for managing a unique subset of the data. Conclusion In conclusion, R is a very powerful tool for the analysis of Java garbage collection log files. The primary difficulty is data cleansing so that information can be read into an R data frame. Once the data has been read into R, a rich set of tools may be used for thorough evaluation.

Read the article

InternalsVisibleTo attribute and security vulnerability

- by Sergey Litvinov

I found one issue with InternalsVisibleTo attribute usage. The idea of InternalsVisibleTo attribute to allow some other assemblies to use internal classes\methods of this assembly. To make it work you need sign your assemblies. So, if other assemblies isn't specified in main assembly and if they have incorrect public key, then they can't use Internal members. But the issue in Reflection Emit type generation. For example, we have CorpLibrary1 assembly and it has such class: public class TestApi { internal virtual void DoSomething() { Console.WriteLine("Base DoSomething"); } public void DoApiTest() { // some internal logic // ... // call internal method DoSomething(); } } This assembly is marked with such attribute to allow another CorpLibrary2 to make inheritor for that TestAPI and override behaviour of DoSomething method. [assembly: InternalsVisibleTo("CorpLibrary2, PublicKey=0024000004800000940000000602000000240000525341310004000001000100434D9C5E1F9055BF7970B0C106AAA447271ECE0F8FC56F6AF3A906353F0B848A8346DC13C42A6530B4ED2E6CB8A1E56278E664E61C0D633A6F58643A7B8448CB0B15E31218FB8FE17F63906D3BF7E20B9D1A9F7B1C8CD11877C0AF079D454C21F24D5A85A8765395E5CC5252F0BE85CFEB65896EC69FCC75201E09795AAA07D0")] The issue is that I'm able to override this internal DoSomething method and break class logic. My steps to do it: Generate new assembly in runtime via AssemblyBuilder Get AssemblyName from CorpLibrary1 and copy PublikKey to new assembly Generate new assembly that will inherit TestApi class As PublicKey and name of generated assembly is the same as in InternalsVisibleTo, then we can generate new DoSomething method that will override internal method in TestAPI assembly Then we have another assembly that isn't related to this CorpLibrary1 and can't use internal members. We have such test code in it: class Program { static void Main(string[] args) { var builder = new FakeBuilder(InjectBadCode, "DoSomething", true); TestApi fakeType = builder.CreateFake(); fakeType.DoApiTest(); // it will display: // Inject bad code // Base DoSomething Console.ReadLine(); } public static void InjectBadCode() { Console.WriteLine("Inject bad code"); } } And this FakeBuilder class has such code: /// /// Builder that will generate inheritor for specified assembly and will overload specified internal virtual method /// /// Target type public class FakeBuilder { private readonly Action _callback; private readonly Type _targetType; private readonly string _targetMethodName; private readonly string _slotName; private readonly bool _callBaseMethod; public FakeBuilder(Action callback, string targetMethodName, bool callBaseMethod) { int randomId = new Random((int)DateTime.Now.Ticks).Next(); _slotName = string.Format("FakeSlot_{0}", randomId); _callback = callback; _targetType = typeof(TFakeType); _targetMethodName = targetMethodName; _callBaseMethod = callBaseMethod; } public TFakeType CreateFake() { // as CorpLibrary1 can't use code from unreferences assemblies, we need to store this Action somewhere. // And Thread is not bad place for that. It's not the best place as it won't work in multithread application, but it's just a sample LocalDataStoreSlot slot = Thread.AllocateNamedDataSlot(_slotName); Thread.SetData(slot, _callback); // then we generate new assembly with the same nameand public key as target assembly trusts by InternalsVisibleTo attribute var newTypeName = _targetType.Name + "Fake"; var targetAssembly = Assembly.GetAssembly(_targetType); AssemblyName an = new AssemblyName(); an.Name = GetFakeAssemblyName(targetAssembly); // copying public key to new generated assembly var assemblyName = targetAssembly.GetName(); an.SetPublicKey(assemblyName.GetPublicKey()); an.SetPublicKeyToken(assemblyName.GetPublicKeyToken()); AssemblyBuilder assemblyBuilder = Thread.GetDomain().DefineDynamicAssembly(an, AssemblyBuilderAccess.RunAndSave); ModuleBuilder moduleBuilder = assemblyBuilder.DefineDynamicModule(assemblyBuilder.GetName().Name, true); // create inheritor for specified type TypeBuilder typeBuilder = moduleBuilder.DefineType(newTypeName, TypeAttributes.Public | TypeAttributes.Class, _targetType); // LambdaExpression.CompileToMethod can be used only with static methods, so we need to create another method that will call our Inject method // we can do the same via ILGenerator, but expression trees are more easy to use MethodInfo methodInfo = CreateMethodInfo(moduleBuilder); MethodBuilder methodBuilder = typeBuilder.DefineMethod(_targetMethodName, MethodAttributes.Public | MethodAttributes.Virtual); ILGenerator ilGenerator = methodBuilder.GetILGenerator(); // call our static method that will call inject method ilGenerator.EmitCall(OpCodes.Call, methodInfo, null); // in case if we need, then we put call to base method if (_callBaseMethod) { var baseMethodInfo = _targetType.GetMethod(_targetMethodName, BindingFlags.NonPublic | BindingFlags.Instance); // place this to stack ilGenerator.Emit(OpCodes.Ldarg_0); // call the base method ilGenerator.EmitCall(OpCodes.Call, baseMethodInfo, new Type[0]); // return ilGenerator.Emit(OpCodes.Ret); } // generate type, create it and return to caller Type cheatType = typeBuilder.CreateType(); object type = Activator.CreateInstance(cheatType); return (TFakeType)type; } /// /// Get name of assembly from InternalsVisibleTo AssemblyName /// private static string GetFakeAssemblyName(Assembly assembly) { var internalsVisibleAttr = assembly.GetCustomAttributes(typeof(InternalsVisibleToAttribute), true).FirstOrDefault() as InternalsVisibleToAttribute; if (internalsVisibleAttr == null) { throw new InvalidOperationException("Assembly hasn't InternalVisibleTo attribute"); } var ind = internalsVisibleAttr.AssemblyName.IndexOf(","); var name = internalsVisibleAttr.AssemblyName.Substring(0, ind); return name; } /// /// Generate such code: /// ((Action)Thread.GetData(Thread.GetNamedDataSlot(_slotName))).Invoke(); /// private LambdaExpression MakeStaticExpressionMethod() { var allocateMethod = typeof(Thread).GetMethod("GetNamedDataSlot", BindingFlags.Static | BindingFlags.Public); var getDataMethod = typeof(Thread).GetMethod("GetData", BindingFlags.Static | BindingFlags.Public); var call = Expression.Call(allocateMethod, Expression.Constant(_slotName)); var getCall = Expression.Call(getDataMethod, call); var convCall = Expression.Convert(getCall, typeof(Action)); var invokExpr = Expression.Invoke(convCall); var lambda = Expression.Lambda(invokExpr); return lambda; } /// /// Generate static class with one static function that will execute Action from Thread NamedDataSlot /// private MethodInfo CreateMethodInfo(ModuleBuilder moduleBuilder) { var methodName = "_StaticTestMethod_" + _slotName; var className = "_StaticClass_" + _slotName; TypeBuilder typeBuilder = moduleBuilder.DefineType(className, TypeAttributes.Public | TypeAttributes.Class); MethodBuilder methodBuilder = typeBuilder.DefineMethod(methodName, MethodAttributes.Static | MethodAttributes.Public); LambdaExpression expression = MakeStaticExpressionMethod(); expression.CompileToMethod(methodBuilder); var type = typeBuilder.CreateType(); return type.GetMethod(methodName, BindingFlags.Static | BindingFlags.Public); } } remarks about sample: as we need to execute code from another assembly, CorpLibrary1 hasn't access to it, so we need to store this delegate somewhere. Just for testing I stored it in Thread NamedDataSlot. It won't work in multithreaded applications, but it's just a sample. I know that we use Reflection to get private\internal members of any class, but within reflection we can't override them. But this issue is allows anyone to override internal class\method if that assembly has InternalsVisibleTo attribute. I tested it on .Net 3.5\4 and it works for both of them. How does it possible to just copy PublicKey without private key and use it in runtime? The whole sample can be found there - https://github.com/sergey-litvinov/Tests_InternalsVisibleTo UPDATE1: That test code in Program and FakeBuilder classes hasn't access to key.sn file and that library isn't signed, so it hasn't public key at all. It just copying it from CorpLibrary1 by using Reflection.Emit

Read the article

Android WebView not loading a JavaScript file, but Android Browser loads it fine.

- by Justin

I'm writing an application which connects to a back office site. The backoffice site contains a whole slew of JavaScript functions, at least 100 times the average site. Unfortunately it does not load them, and causes much of the functionality to not work properly. So I am running a test. I put a page out on my server which loads the FireBugLite javascript text. Its a lot of javascript and perfect to test and see if the Android WebView will load it. The WebView loads nothing, but the browser loads the Firebug Icon. What on earth would make the difference, why can it run in the browser and not in my WebView? Any suggestions. More background information, in order to get the stinking backoffice application available on a Droid (or any other platform except windows) I needed to trick the bakcoffice application to believe what's accessing the website is Internet Explorer. I do this by modifying the WebView User Agent. Also for this application I've slimmed my landing page, so I could give you the source to offer me aid. package ksc.myKMB; import android.app.Activity; import android.app.AlertDialog; import android.app.Dialog; import android.app.ProgressDialog; import android.content.DialogInterface; import android.graphics.Bitmap; import android.os.Bundle; import android.view.Menu; import android.view.MenuInflater; import android.view.MenuItem; import android.view.Window; import android.webkit.WebChromeClient; import android.webkit.WebView; import android.webkit.WebSettings; import android.webkit.WebViewClient; import android.widget.Toast; public class myKMB extends Activity { /** Called when the activity is first created. */ @Override public void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); /** Performs base set up */ /** Create a Activity of this Activity, IE myProcess */ myProcess = this; /*** Create global objects and web browsing objects */ HideDialogOnce = true; webview = new WebView(this) { }; webChromeClient = new WebChromeClient() { public void onProgressChanged(WebView view, int progress) { // Activities and WebViews measure progress with different scales. // The progress meter will automatically disappear when we reach 100% myProcess.setProgress((progress * 100)); //CreateMessage("Progress is : " + progress); } }; webViewClient = new WebViewClient() { public void onReceivedError(WebView view, int errorCode, String description, String failingUrl) { Toast.makeText(myProcess, MessageBegText + description + MessageEndText, Toast.LENGTH_SHORT).show(); } public void onPageFinished (WebView view, String url) { /** Hide dialog */ try { // loadingDialog.dismiss(); } finally { } //myProcess.setProgress(1000); /** Fon't show the dialog while I'm performing fixes */ //HideDialogOnce = true; view.loadUrl("javascript:document.getElementById('JTRANS011').style.visibility='visible';"); } public void onPageStarted(WebView view, String url, Bitmap favicon) { if (HideDialogOnce == false) { //loadingDialog = ProgressDialog.show(myProcess, "", // "One moment, the page is laoding...", true); } else { //HideDialogOnce = true; } } }; getWindow().requestFeature(Window.FEATURE_PROGRESS); webview.setWebChromeClient(webChromeClient); webview.setWebViewClient(webViewClient); setContentView(webview); /** Load the Keynote Browser Settings */ LoadSettings(); webview.loadUrl(LandingPage); } /** Get Menu */ @Override public boolean onCreateOptionsMenu(Menu menu) { MenuInflater inflater = getMenuInflater(); inflater.inflate(R.menu.menu, menu); return true; } /** an item gets pushed */ @Override public boolean onOptionsItemSelected(MenuItem item) { switch (item.getItemId()) { // We have only one menu option case R.id.quit: System.exit(0); break; case R.id.back: webview.goBack(); case R.id.refresh: webview.reload(); case R.id.info: //IncludeJavascript(""); } return true; } /** Begin Globals */ public WebView webview; public WebChromeClient webChromeClient; public WebViewClient webViewClient; public ProgressDialog loadingDialog; public Boolean HideDialogOnce; public Activity myProcess; public String OverideUserAgent_IE = "Mozilla/5.0 (Windows; MSIE 6.0; Android 1.6; en-US) AppleWebKit/525.10+ (KHTML, like Gecko) Version/3.0.4 Safari/523.12.2 myKMB/1.0"; public String LandingPage = "http://kscserver.com/main-leap-slim.html"; public String MessageBegText = "Problem making a connection, Details: "; public String MessageEndText = " For Support Call: (xxx) xxx - xxxx."; public void LoadSettings() { webview.getSettings().setUserAgentString(OverideUserAgent_IE); webview.getSettings().setJavaScriptEnabled(true); webview.getSettings().setBuiltInZoomControls(true); webview.getSettings().setSupportZoom(true); } /** Creates a message alert dialog */ public void CreateMessage(String message) { AlertDialog.Builder builder = new AlertDialog.Builder(this); builder.setMessage(message) .setCancelable(true) .setNegativeButton("Close", new DialogInterface.OnClickListener() { public void onClick(DialogInterface dialog, int id) { dialog.cancel(); } }); AlertDialog alert = builder.create(); alert.show(); } } My Application is running in the background, and as you can see no Firebug in the lower right hand corner. However the browser (the emulator on top) has the same page but shows the firebug. What am I doing wrong? I'm assuming its either not enough memory allocated to the application, process power allocation, or a physical memory thing. I can't tell, all I know is the results are strange. I get the same thing form my android device, the application shows no firebug but the browser shows the firebug.

Read the article

Memory leak involving jQuery Ajax requests

- by Eli Courtwright

I have a webpage that's leaking memory in both IE8 and Firefox; the memory usage displayed in the Windows Process Explorer just keeps growing over time. The following page requests the "unplanned.json" url, which is a static file that never changes (though I do set my Cache-control HTTP header to no-cache to make sure that the Ajax request always goes through). When it gets the results, it clears out an HTML table, loops over the json array it got back from the server, and dynamically adds a row to an HTML table for each entry in the array. Then it waits 2 seconds and repeats this process. Here's the entire webpage: <html> <head> <title>Test Page</title> <script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.3/jquery.min.js"></script> </head> <body> <script type="text/javascript"> function kickoff() { $.getJSON("unplanned.json", resetTable); } function resetTable(rows) { $("#content tbody").empty(); for(var i=0; i<rows.length; i++) { $("<tr>" + "<td>" + rows[i].mpe_name + "</td>" + "<td>" + rows[i].bin + "</td>" + "<td>" + rows[i].request_time + "</td>" + "<td>" + rows[i].filtered_delta + "</td>" + "<td>" + rows[i].failed_delta + "</td>" + "</tr>").appendTo("#content tbody"); } setTimeout(kickoff, 2000); } $(kickoff); </script> <table id="content" border="1" style="width:100% ; text-align:center"> <thead><tr> <th>MPE</th> <th>Bin</th> <th>When</th> <th>Filtered</th> <th>Failed</th> </tr></thead> <tbody></tbody> </table> </body> </html> If it helps, here's an example of the json I'm sending back (it's this exact array wuith thousands of entries instead of just one): [ { mpe_name: "DBOSS-995", request_time: "09/18/2009 11:51:06", bin: 4, filtered_delta: 1, failed_delta: 1 } ] EDIT: I've accepted Toran's extremely helpful answer, but I feel I should post some additional code, since his removefromdom jQuery plugin has some limitations: It only removes individual elements. So you can't give it a query like `$("#content tbody tr")` and expect it to remove all of the elements you've specified. Any element that you remove with it must have an `id` attribute. So if I want to remove my `tbody`, then I must assign an `id` to my `tbody` tag or else it will give an error. It removes the element itself and all of its descendants, so if you simply want to empty that element then you'll have to re-create it afterwards (or modify the plugin to empty instead of remove). So here's my page above modified to use Toran's plugin. For the sake of simplicity I didn't apply any of the general performance advice offered by Peter. Here's the page which now no longer memory leaks: <html> <head> <title>Test Page</title> <script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.3/jquery.min.js"></script> </head> <body> <script type="text/javascript">  </script> <div id="message" style="color:red"></div> <table id="content" border="1" style="width:100% ; text-align:center"> <thead><tr> <th>MPE</th> <th>Bin</th> <th>When</th> <th>Filtered</th> <th>Failed</th> </tr></thead> <tbody id="id_field_required"></tbody> </table> </body> </html> FURTHER EDIT: I'll leave my question unchanged, though it's worth noting that this memory leak has nothing to do with Ajax. In fact, the following code would memory leak just the same and be just as easily solved with Toran's removefromdom jQuery plugin: function resetTable() { $("#content tbody").empty(); for(var i=0; i<1000; i++) { $("#content tbody").append("<tr><td>" + "DBOSS-095" + "</td>" + "<td>" + 4 + "</td>" + "<td>" + "09/18/2009 11:51:06" + "</td>" + "<td>" + 1 + "</td>" + "<td>" + 1 + "</td></tr>"); } setTimeout(resetTable, 2000); } $(resetTable);

Read the article

How do I set the encoding statement in the XML declaration when performing an XSL transformation usi

- by aspiehler

I wrote a simple package installer in WinBatch that needs to update an XML file with information about the package contents. My first stab at it involved loading the file with Msxml2.DOMDocument, adding nodes and data as required, then saving the data back to disk. This worked well enough, except that it would not create tab and CR/LF whitespace in the new data. The solution I came up with was writing an XSL stylesheet that would recreate the XML file with whitespace added back in. I'm doing this by: loading the XSL file into an Msxml2.FreeThreadedDOMDocument object setting that object as the stylesheet property of an Msxml2.XSLTemplate object creating an XSL processor via Msxml2.XSLTemplate.createProcessor() setting my original Msxml2.DOMDocument as the input property of the XSL processor Calling transform() method of the XSL processor, and saving the output to a file. This works as for as reformatting the XML file with tabs and carriage returns, but my XML declaration comes out either as <?xml version="1.0"?> or <?xml version="1.0" encoding="UTF-16"?> depending on whether I used Msxml2.*.6.0 or Msxml2.* objects (a fall back if the system doesn't have 6.0). If the encoding is set to UTF-16, Msxml12.DOMDocument complains about trying to convert UTF-16 to 1-byte encoding the next time I run my package installer. I've tried creating and adding an XML declaration using both createProcessingInstruction() to both the XML and XSL DOM objects, but neither one seems to affect the output of the XSLTemplate processor. I've also set encoding to UTF-8 in the <xsl:output/> tag in my XSL file. Here is the relevant code in my Winbatch script: xmlDoc = ObjectCreate("Msxml2.DOMDocument.6.0") if !xmlDoc then xmlDoc = ObjectCreate("Msxml2.DOMDocument") xmlDoc.async = @FALSE xmlDoc.validateOnParse = @TRUE xmlDoc.resolveExternals = @TRUE xmlDoc.preserveWhiteSpace = @TRUE xmlDoc.setProperty("SelectionLanguge", "XPath") xmlDoc.setProperty("SelectionNamespaces", "xmlns:fns='http://www.abc.com/f_namespace'") xmlDoc.load(xml_file_path) xslStyleSheet = ObjectCreate("Msxml2.FreeThreadedDOMDocument.6.0") if !xslStyleSheet then xslStyleSheet = ObjectCreate("Msxml2.FreeThreadedDOMDocument") xslStyleSheet.async = @FALSE xslStyleSheet.validateOnParse = @TRUE xslStyleSheet.load(xsl_style_sheet_path) xslTemplate = ObjectCreate("Msxml2.XSLTemplate.6.0") if !xslTemplate then xslTemplate = ObjectCreate("Msxml2.XSLTemplate") xslTemplate.stylesheet = xslStyleSheet processor = xslTemplate.createProcessor() processor.input = xmlDoc processor.transform() ; create a new file and write the XML processor output to it fh = FileOpen(output_file_path, "WRITE" , @FALSE) FileWrite(fh, processor.output) FileClose(fh) The style sheet, with some slight changes to protect the innocent: <?xml version="1.0" encoding="UTF-8"?> <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.1"> <xsl:output method="xml" indent="yes" encoding="UTF-8"/> <xsl:template match="/"> <fns:test_station xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:fns="http://www.abc.com/f_namespace"> <xsl:for-each select="/fns:test_station/identification"> <xsl:text>
 </xsl:text> <identification> <xsl:for-each select="./*"> <xsl:text>
 </xsl:text> <xsl:copy-of select="."/> </xsl:for-each> <xsl:text>
 </xsl:text> </identification> </xsl:for-each> <xsl:for-each select="/fns:test_station/software"> <xsl:text>
 </xsl:text> <software> <xsl:for-each select="./package"> <xsl:text>
 </xsl:text> <package> <xsl:for-each select="./*"> <xsl:text>
 </xsl:text> <xsl:copy-of select="."/> </xsl:for-each> <xsl:text>
 </xsl:text> </package> </xsl:for-each> <xsl:text>
 </xsl:text> </software> </xsl:for-each> <xsl:for-each select="/fns:test_station/calibration"> <xsl:text>
 </xsl:text> <calibration> <xsl:for-each select="./item"> <xsl:text>
 </xsl:text> <item> <xsl:for-each select="./*"> <xsl:text>
 </xsl:text> <xsl:copy-of select="."/> </xsl:for-each> <xsl:text>
 </xsl:text> </item> </xsl:for-each> <xsl:text>
 </xsl:text> </calibration> </xsl:for-each> </fns:test_station> </xsl:template> </xsl:stylesheet> And this is a sample output file: <?xml version="1.0" encoding="UTF-16"?> <fns:test_station xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:fns="http://www.abc.com/f_namespace"> <software> <package> <part_number>123456789</part_number> <version>00</version> <test_category>1</test_category> <description>name of software package</description> <execution_path>c:\program files\test\test.exe</execution_path> <execution_arguments>arguments</execution_arguments> <crc_path>c:\ste_config\crc\123456789.lst</crc_path> <uninstall_path>c:\ste_config\uninstall\uninst_123456789.bat</uninstall_path> <install_timestamp>2009-11-09T14:00:44</install_timestamp> </package> </software> </fns:test_station>

Read the article

Hibernate/Spring: failed to lazily initialize - no session or session was closed

- by Niko

I know something similar has been asked already, but unfortunately I wasn't able to find a reliable answer - even with searching for over 2 days. The basic problem is the same as asked multiple time. I have a simple program with two POJOs Event and User - where a user can have multiple events. @Entity @Table public class Event { private Long id; private String name; private User user; @Column @Id @GeneratedValue public Long getId() {return id;} public void setId(Long id) { this.id = id; } @Column public String getName() {return name;} public void setName(String name) {this.name = name;} @ManyToOne @JoinColumn(name="user_id") public User getUser() {return user;} public void setUser(User user) {this.user = user;} } @Entity @Table public class User { private Long id; private String name; private List events; @Column @Id @GeneratedValue public Long getId() { return id; } public void setId(Long id) { this.id = id; } @Column public String getName() { return name; } public void setName(String name) { this.name = name; } @OneToMany(mappedBy="user", fetch=FetchType.LAZY) public List getEvents() { return events; } public void setEvents(List events) { this.events = events; } } Note: This is a sample project. I really want to use Lazy fetching here. I use spring and hibernate and have a simple basic-db.xml for loading: <?xml version="1.0" encoding="UTF-8"? <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:aop="http://www.springframework.org/schema/aop" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-3.0.xsd" <bean id="myDataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close" scope="thread" <property name="driverClassName" value="com.mysql.jdbc.Driver" / <property name="url" value="jdbc:mysql://192.168.1.34:3306/hibernateTest" / <property name="username" value="root" / <property name="password" value="" / <aop:scoped-proxy/ </bean <bean class="org.springframework.beans.factory.config.CustomScopeConfigurer" <property name="scopes" <map <entry key="thread" <bean class="org.springframework.context.support.SimpleThreadScope" / </entry </map </property </bean <bean id="mySessionFactory" class="org.springframework.orm.hibernate3.annotation.AnnotationSessionFactoryBean" scope="thread" <property name="dataSource" ref="myDataSource" / <property name="annotatedClasses" <list <valuedata.model.User</value <valuedata.model.Event</value </list </property <property name="hibernateProperties" <props <prop key="hibernate.dialect"org.hibernate.dialect.MySQLDialect</prop <prop key="hibernate.show_sql"true</prop <prop key="hibernate.hbm2ddl.auto"create</prop </props </property <aop:scoped-proxy/ </bean <bean id="myUserDAO" class="data.dao.impl.UserDaoImpl" <property name="sessionFactory" ref="mySessionFactory" / </bean <bean id="myEventDAO" class="data.dao.impl.EventDaoImpl" <property name="sessionFactory" ref="mySessionFactory" / </bean </beans Note: I played around with the CustomScopeConfigurer and SimpleThreadScope, but that didnt change anything. I have a simple dao-impl (only pasting the userDao - the EventDao is pretty much the same - except with out the "listWith" function: public class UserDaoImpl implements UserDao{ private HibernateTemplate hibernateTemplate; public void setSessionFactory(SessionFactory sessionFactory) { this.hibernateTemplate = new HibernateTemplate(sessionFactory); } @SuppressWarnings("unchecked") @Override public List listUser() { return hibernateTemplate.find("from User"); } @Override public void saveUser(User user) { hibernateTemplate.saveOrUpdate(user); } @Override public List listUserWithEvent() { List users = hibernateTemplate.find("from User"); for (User user : users) { System.out.println("LIST : " + user.getName() + ":"); user.getEvents().size(); } return users; } } I am getting the org.hibernate.LazyInitializationException - failed to lazily initialize a collection of role: data.model.User.events, no session or session was closed at the line with user.getEvents().size(); And last but not least here is the Test class I use: public class HibernateTest { public static void main(String[] args) { ClassPathXmlApplicationContext ac = new ClassPathXmlApplicationContext("basic-db.xml"); UserDao udao = (UserDao) ac.getBean("myUserDAO"); EventDao edao = (EventDao) ac.getBean("myEventDAO"); System.out.println("New user..."); User user = new User(); user.setName("test"); Event event1 = new Event(); event1.setName("Birthday1"); event1.setUser(user); Event event2 = new Event(); event2.setName("Birthday2"); event2.setUser(user); udao.saveUser(user); edao.saveEvent(event1); edao.saveEvent(event2); List users = udao.listUserWithEvent(); System.out.println("Events for users"); for (User u : users) { System.out.println(u.getId() + ":" + u.getName() + " --"); for (Event e : u.getEvents()) { System.out.println("\t" + e.getId() + ":" + e.getName()); } } ((ConfigurableApplicationContext)ac).close(); } } and here is the Exception I get: 1621 [main] ERROR org.hibernate.LazyInitializationException - failed to lazily initialize a collection of role: data.model.User.events, no session or session was closed org.hibernate.LazyInitializationException: failed to lazily initialize a collection of role: data.model.User.events, no session or session was closed at org.hibernate.collection.AbstractPersistentCollection.throwLazyInitializationException(AbstractPersistentCollection.java:380) at org.hibernate.collection.AbstractPersistentCollection.throwLazyInitializationExceptionIfNotConnected(AbstractPersistentCollection.java:372) at org.hibernate.collection.AbstractPersistentCollection.readSize(AbstractPersistentCollection.java:119) at org.hibernate.collection.PersistentBag.size(PersistentBag.java:248) at data.dao.impl.UserDaoImpl.listUserWithEvent(UserDaoImpl.java:38) at HibernateTest.main(HibernateTest.java:44) Exception in thread "main" org.hibernate.LazyInitializationException: failed to lazily initialize a collection of role: data.model.User.events, no session or session was closed at org.hibernate.collection.AbstractPersistentCollection.throwLazyInitializationException(AbstractPersistentCollection.java:380) at org.hibernate.collection.AbstractPersistentCollection.throwLazyInitializationExceptionIfNotConnected(AbstractPersistentCollection.java:372) at org.hibernate.collection.AbstractPersistentCollection.readSize(AbstractPersistentCollection.java:119) at org.hibernate.collection.PersistentBag.size(PersistentBag.java:248) at data.dao.impl.UserDaoImpl.listUserWithEvent(UserDaoImpl.java:38) at HibernateTest.main(HibernateTest.java:44) Things I tried but did not work: assign a threadScope and using beanfactory (I used "request" or "thread" - no difference noticed): // scope stuff Scope threadScope = new SimpleThreadScope(); ConfigurableListableBeanFactory beanFactory = ac.getBeanFactory(); beanFactory.registerScope("request", threadScope); ac.refresh(); ... Setting up a transaction by getting the session object from the deo: ... Transaction tx = ((UserDaoImpl)udao).getSession().beginTransaction(); tx.begin(); users = udao.listUserWithEvent(); ... getting a transaction within the listUserWithEvent() public List listUserWithEvent() { SessionFactory sf = hibernateTemplate.getSessionFactory(); Session s = sf.openSession(); Transaction tx = s.beginTransaction(); tx.begin(); List users = hibernateTemplate.find("from User"); for (User user : users) { System.out.println("LIST : " + user.getName() + ":"); user.getEvents().size(); } tx.commit(); return users; } I am really out of ideas by now. Also, using the listUser or listEvent just work fine.

Read the article

Remove redundant xml namespaces from soapenv:Body

- by drachenstern

If you can tell me the magic google term that instantly gives me clarification, that would be helpful. Here's the part that's throwing an issue when I try to manually deserialize from a string: xsi:type="ns1:errorObject" xmlns:ns1="http://www.example.org/Version_3.0" xsi:type="ns2:errorObject" xmlns:ns2="http://www.example.org/Version_3.0" xsi:type="ns3:errorObject" xmlns:ns3="http://www.example.org/Version_3.0" Here's how I'm deserializing by hand to test it: (in an aspx.cs page with a label on the front to display the value in that I can verify by reading source) (second block of XML duplicates the first but without the extra namespaces) using System; using System.IO; using System.Text; using System.Xml; using System.Xml.Serialization; public partial class test : System.Web.UI.Page { protected void Page_Load(object sender, EventArgs e) { string sourceXml = @"<?xml version=""1.0""?> <InitiateActivityResponse xmlns=""http://www.example.org/Version_3.0""> <InitiateActivityResult> <errorObject errorString=""string 1"" eventTime=""2010-05-21T21:19:15.775Z"" nounType=""Object"" objectID=""object1"" xsi:type=""ns1:errorObject"" xmlns:ns1=""http://www.example.org/Version_3.0"" /> <errorObject errorString=""string 2"" eventTime=""2010-05-21T21:19:15.791Z"" nounType=""Object"" objectID=""object2"" xsi:type=""ns2:errorObject"" xmlns:ns2=""http://www.example.org/Version_3.0"" /> <errorObject errorString=""string 3"" eventTime=""2010-05-21T21:19:15.806Z"" nounType=""Object"" objectID=""object3"" xsi:type=""ns3:errorObject"" xmlns:ns3=""http://www.example.org/Version_3.0"" /> </InitiateActivityResult> </InitiateActivityResponse> "; sourceXml = @"<?xml version=""1.0""?> <InitiateActivityResponse xmlns=""http://www.example.org/Version_3.0""> <InitiateActivityResult> <errorObject errorString=""string 1"" eventTime=""2010-05-21T21:19:15.775Z"" nounType=""Object"" objectID=""object1"" /> <errorObject errorString=""string 2"" eventTime=""2010-05-21T21:19:15.791Z"" nounType=""Object"" objectID=""object2"" /> <errorObject errorString=""string 3"" eventTime=""2010-05-21T21:19:15.806Z"" nounType=""Object"" objectID=""object3"" /> </InitiateActivityResult> </InitiateActivityResponse> "; InitiateActivityResponse fragment = new InitiateActivityResponse(); Type t = typeof( InitiateActivityResponse ); StringBuilder sb = new StringBuilder(); TextWriter textWriter = new StringWriter( sb ); TextReader textReader = new StringReader( sourceXml ); XmlTextReader xmlTextReader = new XmlTextReader( textReader ); XmlSerializer xmlSerializer = new XmlSerializer( t ); object obj = xmlSerializer.Deserialize( xmlTextReader ); fragment = (InitiateActivityResponse)obj; xmlSerializer.Serialize( textWriter, fragment ); //I have a field on my public page that I write to from sb.ToString(); } } Consuming a webservice, I have a class like thus: (all examples foreshortened to as little as possible to show the problem, if boilerplate is missing, my apologies) (this is where I think I want to remove the troublespot) [System.Diagnostics.DebuggerStepThroughAttribute()] [System.ComponentModel.DesignerCategoryAttribute( "code" )] [System.Web.Services.WebServiceBindingAttribute( Name = "MyServerSoapSoapBinding", Namespace = "http://www.example.org/Version_3.0" )] public partial class MyServer : System.Web.Services.Protocols.SoapHttpClientProtocol { public MsgHeader msgHeader { get; set; } public MyServer () { this.Url = "localAddressOmittedOnPurpose"; } [System.Web.Services.Protocols.SoapHeaderAttribute( "msgHeader" )] [System.Web.Services.Protocols.SoapDocumentMethodAttribute( "http://www.example.org/Version_3.0/InitiateActivity", RequestNamespace = "http://www.example.org/Version_3.0", ResponseNamespace = "http://www.example.org/Version_3.0", Use = System.Web.Services.Description.SoapBindingUse.Literal, ParameterStyle = System.Web.Services.Protocols.SoapParameterStyle.Wrapped )] [return: System.Xml.Serialization.XmlElementAttribute( "InitiateActivityResponse" )] public InitiateActivityResponse InitiateActivity(string inputVar) { object[] results = Invoke( "InitiateActivity", new object[] { inputVar } ); return ( (InitiateActivityResponse)( results[0] ) ); } } Class descriptions [System.SerializableAttribute] [System.Diagnostics.DebuggerStepThroughAttribute] [System.ComponentModel.DesignerCategoryAttribute( "code" )] [XmlType( Namespace = "http://www.example.org/Version_3.0", TypeName = "InitiateActivityResponse" )] [XmlRoot( Namespace = "http://www.example.org/Version_3.0" )] public class InitiateActivityResponse { [XmlArray( ElementName = "InitiateActivityResult", IsNullable = true )] [XmlArrayItem( ElementName = "errorObject", IsNullable = false )] public errorObject[] errorObject { get; set; } } [System.SerializableAttribute] [System.Diagnostics.DebuggerStepThroughAttribute] [System.ComponentModel.DesignerCategoryAttribute( "code" )] [XmlTypeAttribute( Namespace = "http://www.example.org/Version_3.0" )] public class errorObject { private string _errorString; private System.DateTime _eventTime; private bool _eventTimeSpecified; private string _nounType; private string _objectID; [XmlAttributeAttribute] public string errorString { get { return _errorString; } set { _errorString = value; } } [XmlAttributeAttribute] public System.DateTime eventTime { get { return _eventTime; } set { _eventTime = value; } } [XmlIgnoreAttribute] public bool eventTimeSpecified { get { return _eventTimeSpecified; } set { _eventTimeSpecified = value; } } [XmlAttributeAttribute] public string nounType { get { return _nounType; } set { _nounType = value; } } [XmlAttributeAttribute] public string objectID { get { return _objectID; } set { _objectID = value; } } } SOAP as it's being received (as seen by Fiddler2) <?xml version="1.0" encoding="utf-8"?> <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <soapenv:Header> <MsgHeader soapenv:actor="http://schemas.xmlsoap.org/soap/actor/next" soapenv:mustUnderstand="0" AppName="AppName" AppVersion="1.0" Company="Company" Pwd="" UserID="" xmlns="http://www.example.org/Version_3.0"/> </soapenv:Header> <soapenv:Body> <InitiateActivityResponse xmlns="http://www.example.org/Version_3.0"> <InitiateActivityResult> <errorObject errorString="Explanatory string for request 1" eventTime="2010-05-24T21:21:37.477Z" nounType="Object" objectID="12345" xsi:type="ns1:errorObject" xmlns:ns1="http://www.example.org/Version_3.0"/> <errorObject errorString="Explanatory string for request 2" eventTime="2010-05-24T21:21:37.493Z" nounType="Object" objectID="45678" xsi:type="ns2:errorObject" xmlns:ns2="http://www.example.org/Version_3.0"/> <errorObject errorString="Explanatory string for request 3" eventTime="2010-05-24T21:21:37.508Z" nounType="Object" objectID="98765" xsi:type="ns3:errorObject" xmlns:ns3="http://www.example.org/Version_3.0"/> </InitiateActivityResult> </InitiateActivityResponse> </soapenv:Body> </soapenv:Envelope> Okay, what should I have not omitted? No I won't post the WSDL, it's hosted behind a firewall, for a vertical stack product. No, I can't change the data sender. Is this somehow automagically handled elsewhere and I just don't know what I don't know? I think I want to do some sort of message sink leading into this method, to intercept the soapenv:Body, but obviously this is for errors, so I'm not going to get errors every time. I'm not entirely sure how to handle this, but some pointers would be nice.

Read the article

Spring & hibernate configuration (using maven): java.lang.ClassNotFoundException: org.hibernate.cfg.

- by Marcos Carceles

Hi, I am trying to include spring and hibernate in an application running on a Weblogic 10.3 server. When I run the application in the server, while accessing an TestServlet to check my configuration I get the following exception: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'mySessionFactory' defined in class path resource [spring-config/HorizonModelPeopleConnectionsSpringContext.xml]: Instantiation of bean failed; nested exception is org.springframework.beans.BeanInstantiationException: Could not instantiate bean class [org.springframework.orm.hibernate3.LocalSessionFactoryBean]: Constructor threw exception; nested exception is java.lang.NoClassDefFoundError: org.hibernate.cfg.Configuration at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:448) at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:251) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:156) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:248) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:160) at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:284) at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:352) at org.springframework.context.support.ClassPathXmlApplicationContext.(ClassPathXmlApplicationContext.java:91) at org.springframework.context.support.ClassPathXmlApplicationContext.(ClassPathXmlApplicationContext.java:75) at org.springframework.context.support.ClassPathXmlApplicationContext.(ClassPathXmlApplicationContext.java:65) at view.com.horizon.test.SpringHibernateServlet.doGet(SpringHibernateServlet.java:27) at javax.servlet.http.HttpServlet.service(HttpServlet.java:707) at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227) at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125) at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:292) at weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:26) at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:56) at oracle.security.wls.filter.SSOSessionSynchronizationFilter.doFilter(SSOSessionSynchronizationFilter.java:279) at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:56) at oracle.dms.wls.DMSServletFilter.doFilter(DMSServletFilter.java:326) at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:56) at weblogic.servlet.internal.RequestEventsFilter.doFilter(RequestEventsFilter.java:27) at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:56) at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3592) at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321) at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:121) at weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2202) at weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2108) at weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1432) at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201) at weblogic.work.ExecuteThread.run(ExecuteThread.java:173) Caused by: org.springframework.beans.BeanInstantiationException: Could not instantiate bean class [org.springframework.orm.hibernate3.LocalSessionFactoryBean]: Constructor threw exception; nested exception is java.lang.NoClassDefFoundError: org.hibernate.cfg.Configuration at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:100) at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:61) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateBean(AbstractAutowireCapableBeanFactory.java:756) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:721) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:384) ... 31 more Caused by: java.lang.NoClassDefFoundError: org.hibernate.cfg.Configuration at org.springframework.orm.hibernate3.LocalSessionFactoryBean.class$(LocalSessionFactoryBean.java:158) at org.springframework.orm.hibernate3.LocalSessionFactoryBean.(LocalSessionFactoryBean.java:158) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:85) ... 35 more I have checked my application and the hibernate jar file is included and it contains the class it says its missing: org.hibernate.cfg.Configuration. The application is built with maven. These are the dependencies of the JAR file using spring and hibernate:   <dependency> <groupId>org.hibernate</groupId> <artifactId>hibernate</artifactId> <version>3.2.7.ga</version> </dependency>  <dependency> <groupId>org.slf4j</groupId> <artifactId>slf4j-log4j12</artifactId> <version>1.5.2</version> </dependency>  <dependency> <groupId>javassist</groupId> <artifactId>javassist</artifactId> <version>3.4.GA</version> </dependency>  <dependency> <groupId>org.springframework</groupId> <artifactId>spring-orm</artifactId> <version>2.5.6</version> </dependency> At first I thought it could be an issue with the versions in the spring and hibernate libraries, so I have tried with different ones, but still I couldn't find anywhere where it says which library versions are compatible,. just got that Spring 2.5.x needs hibernate =3.1 And this is my Spring config file: <?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd"> <bean id="myDataSource" class="org.springframework.jndi.JndiObjectFactoryBean"> <property name="jndiName"> <value>jdbc/WebCenterDS</value> </property>  </bean> <bean id="mySessionFactory" class="org.springframework.orm.hibernate3.LocalSessionFactoryBean"> <property name="dataSource" ref="myDataSource"/> <property name="configLocation"> <value>classpath:hibernate-config/hibernate.cfg.xml</value> </property> <property name="mappingResources"> <list> <value>classpath:com/horizon/model/peopleconnections/profile/internal/bean/CustomAttribute.hbm.xml</value> </list> </property> <property name="hibernateProperties"> <value>hibernate.dialect=org.hibernate.dialect.HSQLDialect</value> </property> </bean> <bean id="profileExtensionDAO" class="com.horizon.model.peopleconnections.profile.internal.dao.ProfileExtensionDAOImpl"> <property name="sessionFactory" ref="mySessionFactory"/> </bean> </beans> The WAR structure I get is the following: J2EETestApplication ¦ springhibernate.jsp ¦ +---WEB-INF ¦ faces-config.xml ¦ web.xml ¦ weblogic.xml ¦ +---classes ¦ +---view ¦ +---com ¦ +---horizon ¦ +---test ¦ SpringHibernateServlet.class ¦ +---lib activation-1.1.jar antlr-2.7.6.jar aopalliance-1.0.jar asm-1.5.3.jar asm-attrs-1.5.3.jar cglib-2.1_3.jar commons-codec-1.3.jar commons-collections-2.1.1.jar commons-logging-1.1.1.jar dom4j-1.6.1.jar ehcache-1.2.3.jar hibernate-3.2.7.ga.jar horizon-model-commons-1.0-SNAPSHOT.jar horizon-model-peopleconnections-1.0-SNAPSHOT.jar horizon-shared-commons-1.0-SNAPSHOT.jar horizon-shared-logging-1.0-SNAPSHOT.jar horizon-shared-util-1.0-SNAPSHOT.jar horizon-shared-webcenter-1.0-SNAPSHOT.jar horizon-shared-webcenter.jar httpclient-4.0.1.jar httpcore-4.0.1.jar javassist-3.4.GA.jar jta-1.0.1B.jar log4j-1.2.14.jar mail-1.4.1.jar peopleconnections-profile-model-11.1.1.2.0.jar saxon-9.1.0.8.jar serviceframework-11.1.1.2.0.jar slf4j-api-1.5.2.jar slf4j-log4j12-1.5.2.jar spring-beans-2.5.6.jar spring-context-2.5.6.jar spring-core-2.5.6.jar spring-orm-2.5.6.jar spring-tx-2.5.6.jar Is there any dependency or configuration I am missing? If I use hibernate without spring I don't get the ClassDefNotFoundException.

Read the article

How to group using XSLT

- by AdRock

I'm having trouble grouping a set of nodes. I've found an article that does work with grouping and i have tested it and it works on a small test stylesheet i have I now need to use it in my stylesheet where I only want to select node sets that have a specific value. What I want to do in my stylesheet is select all users who have a userlevel of 2 then to group them by the volunteer region. What happens at the minute is that it gets the right amount of users with userlevel 2 but doesn't print them. It just repeats the first user in the xml file. <?xml version="1.0" encoding="ISO-8859-1"?> <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"> <xsl:key name="volunteers-by-region" match="volunteer" use="region" /> <xsl:template name="hoo" match="/"> <html> <head> <title>Registered Volunteers</title> <link rel="stylesheet" type="text/css" href="volunteer.css" /> </head> <body> <h1>Registered Volunteers</h1> <h3>Ordered by the username ascending</h3> <xsl:for-each select="folktask/member[user/account/userlevel='2']"> <xsl:for-each select="volunteer[count(. | key('volunteers-by-region', region)[1]) = 1]"> <xsl:sort select="region" /> <xsl:for-each select="key('volunteers-by-region', region)"> <xsl:sort select="folktask/member/user/personal/name" /> <div class="userdiv"> <xsl:call-template name="member_userid"> <xsl:with-param name="myid" select="/folktask/member/user/@id" /> </xsl:call-template> <xsl:call-template name="volunteer_volid"> <xsl:with-param name="volid" select="/folktask/member/volunteer/@id" /> </xsl:call-template> <xsl:call-template name="volunteer_role"> <xsl:with-param name="volrole" select="/folktask/member/volunteer/roles" /> </xsl:call-template> <xsl:call-template name="volunteer_region"> <xsl:with-param name="volloc" select="/folktask/member/volunteer/region" /> </xsl:call-template> </div> </xsl:for-each> </xsl:for-each> </xsl:for-each> <xsl:if test="position()=last()"> <div class="count"><h2>Total number of volunteers: <xsl:value-of select="count(/folktask/member/user/account/userlevel[text()=2])"/></h2></div> </xsl:if> </body> </html> </xsl:template> <xsl:template name="member_userid"> <xsl:param name="myid" select="'Not Available'" /> <div class="heading bold"><h2>USER ID: <xsl:value-of select="$myid" /></h2></div> </xsl:template> <xsl:template name="volunteer_volid"> <xsl:param name="volid" select="'Not Available'" /> <div class="heading2 bold"><h2>VOLUNTEER ID: <xsl:value-of select="$volid" /></h2></div> </xsl:template> <xsl:template name="volunteer_role"> <xsl:param name="volrole" select="'Not Available'" /> <div class="small bold">ROLES:</div> <div class="large"> <xsl:choose> <xsl:when test="string-length($volrole)!=0"> <xsl:value-of select="$volrole" /> </xsl:when> <xsl:otherwise> <xsl:text> </xsl:text> </xsl:otherwise> </xsl:choose> </div> </xsl:template> <xsl:template name="volunteer_region"> <xsl:param name="volloc" select="'Not Available'" /> <div class="small bold">REGION:</div> <div class="large"><xsl:value-of select="$volloc" /></div> </xsl:template> </xsl:stylesheet> here is my full xml file <?xml version="1.0" encoding="ISO-8859-1" ?> <?xml-stylesheet type="text/xsl" href="volunteers.xsl"?> <folktask xmlns:xs="http://www.w3.org/2001/XMLSchema-instance" xs:noNamespaceSchemaLocation="folktask.xsd"> <member> <user id="1"> <personal> <name>Abbie Hunt</name> <sex>Female</sex> <address1>108 Access Road</address1> <address2></address2> <city>Wells</city> <county>Somerset</county> <postcode>BA5 8GH</postcode> <telephone>01528927616</telephone> <mobile>07085252492</mobile> <email>[email protected]</email> </personal> <account> <username>AdRock</username> <password>269eb625e2f0cf6fae9a29434c12a89f</password> <userlevel>4</userlevel> <signupdate>2010-03-26T09:23:50</signupdate> </account> </user> <volunteer id="1"> <roles></roles> <region>South West</region> </volunteer> </member> <member> <user id="2"> <personal> <name>Aidan Harris</name> <sex>Male</sex> <address1>103 Aiken Street</address1> <address2></address2> <city>Chichester</city> <county>Sussex</county> <postcode>PO19 4DS</postcode> <telephone>01905149894</telephone> <mobile>07784467941</mobile> <email>[email protected]</email> </personal> <account> <username>AmbientExpert</username> <password>8e64214160e9dd14ae2a6d9f700004a6</password> <userlevel>2</userlevel> <signupdate>2010-03-26T09:23:50</signupdate> </account> </user> <volunteer id="2"> <roles>Van Driver,gas Fitter</roles> <region>South Central</region> </volunteer> </member> <member> <user id="3"> <personal> <name>Skye Saunders</name> <sex>Female</sex> <address1>31 Anns Court</address1> <address2></address2> <city>Cirencester</city> <county>Gloucestershire</county> <postcode>GL7 1JG</postcode> <telephone>01958303514</telephone> <mobile>07260491667</mobile> <email>[email protected]</email> </personal> <account> <username>BigUndecided</username> <password>ea297847f80e046ca24a8621f4068594</password> <userlevel>2</userlevel> <signupdate>2010-03-26T09:23:50</signupdate> </account> </user> <volunteer id="3"> <roles>Scaffold Erector</roles> <region>South West</region> </volunteer> </member> <member> <user id="4"> <personal> <name>Connor Lawson</name> <sex>Male</sex> <address1>12 Ash Way</address1> <address2></address2> <city>Swindon</city> <county>Wiltshire</county> <postcode>SN3 6GS</postcode> <telephone>01791928119</telephone> <mobile>07338695664</mobile> <email>[email protected]</email> </personal> <account> <username>iTuneStinker</username> <password>3a1f5fda21a07bfff20c41272bae7192</password> <userlevel>3</userlevel> <signupdate>2010-03-26T09:23:50</signupdate> </account> </user> <festival id="1"> <event> <eventname>Oxford Folk Festival</eventname> <url>http://www.oxfordfolkfestival.com/</url> <datefrom>2010-04-07</datefrom> <dateto>2010-04-09</dateto> <location>Oxford</location> <eventpostcode>OX1 9BE</eventpostcode> <additional>Oxford Folk Festival is going into it's third year in 2006. As well as needing volunteers to steward for the event on the weekend itself, we would be delighted to hear from people willing to help in year round festival work such as stuffing envelopes for mailings, poster and leaflet distribution, and stewarding duties at festival pre-events.</additional> <coords> <lat>51.735640</lat> <lng>-1.276136</lng> </coords> </event> <contact> <conname>Stuart Vincent</conname> <conaddress1>P.O. Box 642</conaddress1> <conaddress2></conaddress2> <concity>Oxford</concity> <concounty>Bedfordshire</concounty> <conpostcode>OX1 3BY</conpostcode> <contelephone>01865 79073</contelephone> <conmobile></conmobile> <fax></fax> <conemail>[email protected]</conemail> </contact> </festival> </member> <member> <user id="5"> <personal> <name>Lewis King</name> <sex>Male</sex> <address1>67 Arbors Way</address1> <address2></address2> <city>Sherborne</city> <county>Dorset</county> <postcode>DT9 0GS</postcode> <telephone>01446139701</telephone> <mobile>07292614033</mobile> <email>[email protected]</email> </personal> <account> <username>Runninglife</username> <password>98fab0a27c34ddb2b0618bc184d4331d</password> <userlevel>2</userlevel> <signupdate>2010-03-26T09:23:50</signupdate> </account> </user> <volunteer id="4"> <roles>Van Driver</roles> <region>South West</region> </volunteer> </member> <member> <user id="6"> <personal> <name>Cameron Lee</name> <sex>Male</sex> <address1>77 Arrington Road</address1> <address2></address2> <city>Solihull</city> <county>Warwickshire</county> <postcode>B90 6FG</postcode> <telephone>01435158624</telephone> <mobile>07789503179</mobile> <email>[email protected]</email> </personal> <account> <username>love2Mixer</username> <password>1df752d54876928639cea07ce036a9c0</password> <userlevel>2</userlevel> <signupdate>2010-03-26T09:23:50</signupdate> </account> </user> <volunteer id="5"> <roles>Fire Warden</roles> <region>Midlands</region> </volunteer> </member> <member> <user id="7"> <personal> <name>Lexie Dean</name> <sex>Female</sex> <address1>38 Bloomfield Court</address1> <address2></address2> <city>Windermere</city> <county>Westmorland</county> <postcode>LA23 8BM</postcode> <telephone>01781207188</telephone> <mobile>07127461231</mobile> <email>[email protected]</email> </personal> <account> <username>MailNetworker</username> <password>0e070701839e612bf46af4421db4f44b</password> <userlevel>3</userlevel> <signupdate>2010-03-26T09:23:50</signupdate> </account> </user> <festival id="2"> <event> <eventname>Middlewich Folk And Boat Festival</eventname> <url>http://midfest.org.uk/mfab/</url> <datefrom>2010-06-16</datefrom> <dateto>2010-06-18</dateto> <location>Middlewich</location> <eventpostcode>CW10 9BX</eventpostcode> <additional>We welcome stewards staying on campsite or boats.</additional> <coords> <lat>53.190562</lat> <lng>-2.444926</lng> </coords> </event> <contact> <conname>Festival Committee</conname> <conaddress1>PO Box 141</conaddress1> <conaddress2></conaddress2> <concity>Winsford</concity> <concounty>Cheshire</concounty> <conpostcode>CW10 9WB</conpostcode> <contelephone>07092 39050</contelephone> <conmobile>07092 39050</conmobile> <fax></fax> <conemail>[email protected]</conemail> </contact> </festival> </member> <member> <user id="8"> <personal> <name>Liam Chapman</name> <sex>Male</sex> <address1>99 Black Water Drive</address1> <address2></address2> <city>St.Austell</city> <county>Cornwall</county> <postcode>PL25 3GF</postcode> <telephone>01835629418</telephone> <mobile>07695179069</mobile> <email>[email protected]</email> </personal> <account> <username>GreenWimp</username> <password>1fe3df99a841dc4f723d21af89e0990f</password> <userlevel>1</userlevel> <signupdate>2010-03-26T09:23:50</signupdate> </account> </user> </member> <member> <user id="9"> <personal> <name>Brandon Harrison</name> <sex>Male</sex> <address1>41 Arlington Way</address1> <address2></address2> <city>Dorchester</city> <county>Dorset</county> <postcode>DT1 3JS</postcode> <telephone>01293626735</telephone> <mobile>07277145760</mobile> <email>[email protected]</email> </personal> <account> <username>LovelyStar</username> <password>8b53b66f323aa5e6a083edb4fd44456b</password> <userlevel>1</userlevel> <signupdate>2010-03-26T09:23:50</signupdate> </account> </user> </member> <member> <user id="10"> <personal> <name>Samuel Young</name> <sex>Male</sex> <address1>102 Bailey Hill Road</address1> <address2></address2> <city>Wolverhampton</city> <county>Staffordshire</county> <postcode>WV7 8HS</postcode> <telephone>01594531382</telephone> <mobile>07544663654</mobile> <email>[email protected]</email> </personal> <account> <username>GuruSassy</username> <password>00da02da6c143c3d136bf60b8bfcf43e</password> <userlevel>2</userlevel> <signupdate>2010-03-26T09:23:50</signupdate> </account> </user> <volunteer id="6"> <roles>Fire Warden</roles> <region>Midlands</region> </volunteer> </member> <member> <user id="11"> <personal> <name>Alexander Harris</name> <sex>Male</sex> <address1>93 Beguine Drive</address1> <address2></address2> <city>Winchester</city> <county>Hampshire</county> <postcode>S23 2FD</postcode> <telephone>01452496582</telephone> <mobile>07353867291</mobile> <email>[email protected]</email> </personal> <account> <username>GuitarExpert</username> <password>0102ad3740028e155925e9918ead3bde</password> <userlevel>2</userlevel> <signupdate>2010-03-26T09:23:50</signupdate> </account> </user> <volunteer id="7"> <roles>Scaffold Erector</roles> <region>North East</region> </volunteer> </member> <member> <user id="12"> <personal> <name>Tyler Mcdonald</name> <sex>Male</sex> <address1>44 Baker Road</address1> <address2></address2> <city>Bromley</city> <county>Kent</county> <postcode>BR1 2GD</postcode> <telephone>01918704546</telephone> <mobile>07314062451</mobile> <email>[email protected]</email> </personal> <account> <username>WildWish</username> <password>073220bb5e9a12ad202bb7d94dcc86f7</password> <userlevel>1</userlevel> <signupdate>2010-03-26T09:23:50</signupdate> </account> </user> </member> <member> <user id="13"> <personal> <name>Skye Mason</name> <sex>Female</sex> <address1>56 Cedar Creek Church Road</address1> <address2></address2> <city>Bracknell</city> <county>Berkshire</county> <postcode>RG12 1AQ</postcode> <telephone>01787607618</telephone> <mobile>07540218868</mobile> <email>[email protected]</email> </personal> <account> <username>PizzaDork</username> <password>74c54937ee7051ee7f4ebc11296ed531</password> <userlevel>1</userlevel> <signupdate>2010-03-26T09:23:50</signupdate> </account> </user> </member> <member> <user id="14"> <personal> <name>Maryam Rose</name> <sex>Female</sex> <address1>98 Baptist Circle</address1> <address2></address2> <city>Newbury</city> <county>Berkshire</county> <postcode>RG14 8DF</postcode> <telephone>01691317999</telephone> <mobile>07212477154</mobile> <email>[email protected]</email> </personal> <account> <username>SexTech</username> <password>f1c21f9f1e999da97d7dc460bb876fcf</password> <userlevel>3</userlevel> <signupdate>2010-03-26T09:23:50</signupdate> </account> </user> <festival id="3"> <event> <eventname>Birdsedge Village Festival</eventname> <url>http://www.birdsedge.co.uk/</url> <datefrom>2010-07-08</datefrom> <dateto>2010-07-09</dateto> <location>Birdsedge</location> <eventpostcode>HD8 8XT</eventpostcode> <additional></additional> <coords> <lat>53.565644</lat> <lng>-1.696196</lng> </coords> </event> <contact> <conname>Jacey Bedford</conname> <conaddress1>Penistone Road</conaddress1> <conaddress2>Birdsedge</conaddress2> <concity>Huddersfield</concity> <concounty>West Yorkshire</concounty> <conpostcode>HD8 8XT</conpostcode> <contelephone>01484 60623</contelephone> <conmobile></conmobile> <fax></fax> <conemail>[email protected]</conemail> </contact> </festival> </member> <member> <user id="15"> <personal> <name>Lexie Rogers</name> <sex>Female</sex> <address1>38 Bishop Road</address1> <address2></address2> <city>Matlock</city> <county>Derbyshire</county> <postcode>DE4 1BX</postcode> <telephone>01961168823</telephone> <mobile>07170855351</mobile> <email>[email protected]</email> </personal> <account> <username>ShipBurglar</username> <password>cc190488a95667cb117e20bc6c7c330e</password> <userlevel>2</userlevel> <signupdate>2010-03-26T09:23:50</signupdate> </account> </user> <volunteer id="8"> <roles>Gas Fitter</roles> <region>Midlands</region> </volunteer> </member> <member> <user id="16"> <personal> <name>Noah Parker</name> <sex>Male</sex> <address1>112 Canty Road</address1> <address2></address2> <city>Keswick</city> <county>Cumberland</county> <postcode>CA12 4TR</postcode> <telephone>01931272522</telephone> <mobile>07610026576</mobile> <email>[email protected]</email> </personal> <account> <username>AwsomeMoon</username> <password>50b770539bdf08543f15778fc7a6f188</password> <userlevel>2</userlevel> <signupdate>2010-03-26T09:23:50</signupdate> </account> </user> <volunteer id="9"> <roles>Van Driver</roles> <region>North West</region> </volunteer> </member> <member> <user id="17"> <personal> <name>Elliot Mitchell</name> <sex>Male</sex> <address1>102 Brown Loop</address1> <address2></address2> <city>Grimsby</city> <county>Lincolnshire</county> <postcode>OX16 4QP</postcode> <telephone>01212971319</telephone> <mobile>07544663654</mobile> <email>[email protected]</email> </personal> <account> <username>msBasher</username> <password>c38fad85badcdff6e3559ef38656305d</password> <userlevel>1</userlevel> <signupdate>2010-03-26T09:23:50</signupdate> </account> </user> </member> <member> <user id="18"> <personal> <name>Scarlett Rose</name> <sex>Female</sex> <address1>93 Cedar Lane</address1> <address2></address2> <city>Stourbridge</city> <county>Warminster</county> <postcode>DY8 4NX</postcode> <telephone>01537477435</telephone> <mobile>07353867291</mobile> <email>[email protected]</email> </personal> <account> <username>MakeupWimp</username> <password>16a9b7910fc34304c1d1a6a1b0c31502</password> <userlevel>1</userlevel> <signupdate>2010-03-26T09:23:50</signupdate> </account> </user> </member> <member> <user id="19"> <personal> <name>Katie Butler</name> <sex>Female</sex> <address1>44 Boulder Crest Road</address1> <address2></address2> <city>Bungay</city> <county>Suffolk</county> <postcode>NR35 1LT</postcode> <telephone>01419124094</telephone> <mobile>07314062451</mobile> <email>[email protected]</email> </personal> <account> <username>TomatoCrunch</username> <password>d7eba53443ec4ddcee69ed71b2023fc0</password> <userlevel>1</userlevel> <signupdate>2010-03-26T09:23:50</signupdate> </account> </user> </member> <member> <user id="20"> <personal> <name>Jayden Richards</name> <sex>Male</sex> <address1>56 Corson Trail</address1> <address2></address2> <city>Sandy</city> <county>Bedfordshire</county> <postcode>SG19 6DF</postcode> <telephone>01882134438</telephone> <mobile>07540218868</mobile> <email>[email protected]</email> </personal> <account> <username>nightmareTwig</username> <password>8a9c08c7b6473493e8a5da15dd541025</password> <userlevel>3</userlevel> <signupdate>2010-03-26T09:23:50</signupdate> </account> </user> <festival id="4"> <event> <eventname>East Barnet Festival</eventname> <url>http://www.eastbarnetfestival.org.uk</url> <datefrom>2010-07-01</datefrom> <dateto>2010-07-03</dateto> <location>East Barnet</location> <eventpostcode>EN4 8TB</eventpostcode> <additional></additional> <coords> <lat>51.641556</lat> <lng>-0.163018</lng> </coords> </event> <contact> <conname>East Barnet Festival Commitee</conname> <conaddress1>Oak Hill Park</conaddress1> <conaddress2>Church Hill Road</conaddress2> <concity>East Barnet</concity> <concounty>Hertfordshire</concounty> <conpostcode>EN4 8TB</conpostcode> <contelephone>07071781745</contelephone> <conmobile>07071781745</conmobile> <fax></fax> <conemail>[email protected]</conemail> </contact> </festival> </member> <member> <user id="21"> <personal> <name>Abbie Jackson</name> <sex>Female</sex> <address1>98 Briarwood Lane</address1> <address2></address2> <city>Weymouth</city> <county>Dorset</county> <postcode>DT3 6TS</postcode> <telephone>01575629969</telephone> <mobile>07212477154</mobile> <email>[email protected]</email> </personal> <account> <username>CrazyBlockhead</username> <password>4ce56fb13d043be605037ace4fbd9fa5</password> <userlevel>2</u

Read the article

Snort's problems in generating alert from Darpa 1998 intrusion detection dataset.

- by manofseven2

Hi. I’m working on DARPA 1998 intrusion detection dataset. When I run snort on this dataset (outside.tcpdump file), snort don’t generate complete list of alerts. It means snort start from last few hours of tcpdump file and generate alerts about this section of file and all of packets in first hours are ignored. Another problem in generatin alert is in time stamp of generated alerts. This means when I run snort on a specific day of dataset, snort insert incorrect time stamp for that alert. The configuration and command line statement and other information about my research are: Snort version: 2.8.6 Operating system: windows XP Rule version: snortrules-snapshot-2860_s.tar.gz -———————————————————————— Command line: snort_2.8.6 c D:\programs\Snort_2.8.6\snort\etc\snort.conf -r d:\users\amir\docs\darpa\training_data\week_3\monday\outside.tcpdump -l D:\users\amir\current-task\research\thesis\snort\890230 -————————————————————————— Snort.config Hi. I'm working on DARPA 1998 intrusion detection dataset. When I run snort on this dataset (outside.tcpdump file), snort don't generate complete list of alerts. It means snort start from last few hours of tcpdump file and generate alerts about this section of file and all of packets in first hours are ignored. Another problem in generatin alert is in time stamp of generated alerts. This means when I run snort on a specific day of dataset, snort insert incorrect time stamp for that alert. The configuration and command line statement and other information about my research are: Snort version: 2.8.6 Operating system: windows XP Rule version: snortrules-snapshot-2860_s.tar.gz Command line: snort_2.8.6 -c D:\programs\Snort_2.8.6\snort\etc\snort.conf -r d:\users\amir\docs\darpa\training_data\week_3\monday\outside.tcpdump -l D:\users\amir\current-task\research\thesis\snort\890230 Snort.config # Setup the network addresses you are protecting var HOME_NET any # Set up the external network addresses. Leave as "any" in most situations var EXTERNAL_NET any # List of DNS servers on your network var DNS_SERVERS $HOME_NET # List of SMTP servers on your network var SMTP_SERVERS $HOME_NET # List of web servers on your network var HTTP_SERVERS $HOME_NET # List of sql servers on your network var SQL_SERVERS $HOME_NET # List of telnet servers on your network var TELNET_SERVERS $HOME_NET # List of ssh servers on your network var SSH_SERVERS $HOME_NET # List of ports you run web servers on portvar HTTP_PORTS [80,1220,2301,3128,7777,7779,8000,8008,8028,8080,8180,8888,9999] # List of ports you want to look for SHELLCODE on. portvar SHELLCODE_PORTS !80 # List of ports you might see oracle attacks on portvar ORACLE_PORTS 1024: # List of ports you want to look for SSH connections on: portvar SSH_PORTS 22 # other variables, these should not be modified var AIM_SERVERS [64.12.24.0/23,64.12.28.0/23,64.12.161.0/24,64.12.163.0/24,64.12.200.0/24,205.188.3.0/24,205.188.5.0/24,205.188.7.0/24,205.188.9.0/24,205.188.153.0/24,205.188.179.0/24,205.188.248.0/24] var RULE_PATH ../rules var SO_RULE_PATH ../so_rules var PREPROC_RULE_PATH ../preproc_rules # Stop generic decode events: config disable_decode_alerts # Stop Alerts on experimental TCP options config disable_tcpopt_experimental_alerts # Stop Alerts on obsolete TCP options config disable_tcpopt_obsolete_alerts # Stop Alerts on T/TCP alerts config disable_tcpopt_ttcp_alerts # Stop Alerts on all other TCPOption type events: config disable_tcpopt_alerts # Stop Alerts on invalid ip options config disable_ipopt_alerts # Alert if value in length field (IP, TCP, UDP) is greater th elength of the packet # config enable_decode_oversized_alerts # Same as above, but drop packet if in Inline mode (requires enable_decode_oversized_alerts) # config enable_decode_oversized_drops # Configure IP / TCP checksum mode config checksum_mode: all config pcre_match_limit: 1500 config pcre_match_limit_recursion: 1500 # Configure the detection engine See the Snort Manual, Configuring Snort - Includes - Config config detection: search-method ac-split search-optimize max-pattern-len 20 # Configure the event queue. For more information, see README.event_queue config event_queue: max_queue 8 log 3 order_events content_length dynamicpreprocessor directory D:\programs\Snort_2.8.6\snort\lib\snort_dynamicpreprocessor dynamicengine D:\programs\Snort_2.8.6\snort\lib\snort_dynamicengine\sf_engine.dll # path to dynamic rules libraries #dynamicdetection directory /usr/local/lib/snort_dynamicrules preprocessor frag3_global: max_frags 65536 preprocessor frag3_engine: policy windows detect_anomalies overlap_limit 10 min_fragment_length 100 timeout 180 preprocessor stream5_global: max_tcp 8192, track_tcp yes, track_udp yes, track_icmp no preprocessor stream5_tcp: policy windows, detect_anomalies, require_3whs 180, \ overlap_limit 10, small_segments 3 bytes 150, timeout 180, \ ports client 21 22 23 25 42 53 79 109 110 111 113 119 135 136 137 139 143 \ 161 445 513 514 587 593 691 1433 1521 2100 3306 6665 6666 6667 6668 6669 \ 7000 32770 32771 32772 32773 32774 32775 32776 32777 32778 32779, \ ports both 80 443 465 563 636 989 992 993 994 995 1220 2301 3128 6907 7702 7777 7779 7801 7900 7901 7902 7903 7904 7905 \ 7906 7908 7909 7910 7911 7912 7913 7914 7915 7916 7917 7918 7919 7920 8000 8008 8028 8080 8180 8888 9999 preprocessor stream5_udp: timeout 180 preprocessor http_inspect: global iis_unicode_map unicode.map 1252 compress_depth 20480 decompress_depth 20480 preprocessor http_inspect_server: server default \ chunk_length 500000 \ server_flow_depth 0 \ client_flow_depth 0 \ post_depth 65495 \ oversize_dir_length 500 \ max_header_length 750 \ max_headers 100 \ ports { 80 1220 2301 3128 7777 7779 8000 8008 8028 8080 8180 8888 9999 } \ non_rfc_char { 0x00 0x01 0x02 0x03 0x04 0x05 0x06 0x07 } \ enable_cookie \ extended_response_inspection \ inspect_gzip \ apache_whitespace no \ ascii no \ bare_byte no \ directory no \ double_decode no \ iis_backslash no \ iis_delimiter no \ iis_unicode no \ multi_slash no \ non_strict \ u_encode yes \ webroot no preprocessor rpc_decode: 111 32770 32771 32772 32773 32774 32775 32776 32777 32778 32779 no_alert_multiple_requests no_alert_large_fragments no_alert_incomplete preprocessor bo preprocessor ftp_telnet: global inspection_type stateful encrypted_traffic no preprocessor ftp_telnet_protocol: telnet \ ayt_attack_thresh 20 \ normalize ports { 23 } \ detect_anomalies preprocessor ftp_telnet_protocol: ftp server default \ def_max_param_len 100 \ ports { 21 2100 3535 } \ telnet_cmds yes \ ignore_telnet_erase_cmds yes \ ftp_cmds { ABOR ACCT ADAT ALLO APPE AUTH CCC CDUP } \ ftp_cmds { CEL CLNT CMD CONF CWD DELE ENC EPRT } \ ftp_cmds { EPSV ESTA ESTP FEAT HELP LANG LIST LPRT } \ ftp_cmds { LPSV MACB MAIL MDTM MIC MKD MLSD MLST } \ ftp_cmds { MODE NLST NOOP OPTS PASS PASV PBSZ PORT } \ ftp_cmds { PROT PWD QUIT REIN REST RETR RMD RNFR } \ ftp_cmds { RNTO SDUP SITE SIZE SMNT STAT STOR STOU } \ ftp_cmds { STRU SYST TEST TYPE USER XCUP XCRC XCWD } \ ftp_cmds { XMAS XMD5 XMKD XPWD XRCP XRMD XRSQ XSEM } \ ftp_cmds { XSEN XSHA1 XSHA256 } \ alt_max_param_len 0 { ABOR CCC CDUP ESTA FEAT LPSV NOOP PASV PWD QUIT REIN STOU SYST XCUP XPWD } \ alt_max_param_len 200 { ALLO APPE CMD HELP NLST RETR RNFR STOR STOU XMKD } \ alt_max_param_len 256 { CWD RNTO } \ alt_max_param_len 400 { PORT } \ alt_max_param_len 512 { SIZE } \ chk_str_fmt { ACCT ADAT ALLO APPE AUTH CEL CLNT CMD } \ chk_str_fmt { CONF CWD DELE ENC EPRT EPSV ESTP HELP } \ chk_str_fmt { LANG LIST LPRT MACB MAIL MDTM MIC MKD } \ chk_str_fmt { MLSD MLST MODE NLST OPTS PASS PBSZ PORT } \ chk_str_fmt { PROT REST RETR RMD RNFR RNTO SDUP SITE } \ chk_str_fmt { SIZE SMNT STAT STOR STRU TEST TYPE USER } \ chk_str_fmt { XCRC XCWD XMAS XMD5 XMKD XRCP XRMD XRSQ } \ chk_str_fmt { XSEM XSEN XSHA1 XSHA256 } \ cmd_validity ALLO \ cmd_validity EPSV \ cmd_validity MACB \ cmd_validity MDTM \ cmd_validity MODE \ cmd_validity PORT \ cmd_validity PROT \ cmd_validity STRU \ cmd_validity TYPE preprocessor ftp_telnet_protocol: ftp client default \ max_resp_len 256 \ bounce yes \ ignore_telnet_erase_cmds yes \ telnet_cmds yes preprocessor smtp: ports { 25 465 587 691 } \ inspection_type stateful \ normalize cmds \ normalize_cmds { MAIL RCPT HELP HELO ETRN EHLO EXPN VRFY ATRN SIZE BDAT DEBUG EMAL ESAM ESND ESOM EVFY IDENT NOOP RSET SEND SAML SOML AUTH TURN DATA QUIT ONEX QUEU STARTTLS TICK TIME TURNME VERB X-EXPS X-LINK2STATE XADR XAUTH XCIR XEXCH50 XGEN XLICENSE XQUE XSTA XTRN XUSR } \ max_command_line_len 512 \ max_header_line_len 1000 \ max_response_line_len 512 \ alt_max_command_line_len 260 { MAIL } \ alt_max_command_line_len 300 { RCPT } \ alt_max_command_line_len 500 { HELP HELO ETRN EHLO } \ alt_max_command_line_len 255 { EXPN VRFY ATRN SIZE BDAT DEBUG EMAL ESAM ESND ESOM EVFY IDENT NOOP RSET } \ alt_max_command_line_len 246 { SEND SAML SOML AUTH TURN ETRN DATA RSET QUIT ONEX QUEU STARTTLS TICK TIME TURNME VERB X-EXPS X-LINK2STATE XADR XAUTH XCIR XEXCH50 XGEN XLICENSE XQUE XSTA XTRN XUSR } \ valid_cmds { MAIL RCPT HELP HELO ETRN EHLO EXPN VRFY ATRN SIZE BDAT DEBUG EMAL ESAM ESND ESOM EVFY IDENT NOOP RSET SEND SAML SOML AUTH TURN DATA QUIT ONEX QUEU STARTTLS TICK TIME TURNME VERB X-EXPS X-LINK2STATE XADR XAUTH XCIR XEXCH50 XGEN XLICENSE XQUE XSTA XTRN XUSR } \ xlink2state { enabled } preprocessor ssh: server_ports { 22 } \ autodetect \ max_client_bytes 19600 \ max_encrypted_packets 20 \ max_server_version_len 100 \ enable_respoverflow enable_ssh1crc32 \ enable_srvoverflow enable_protomismatch preprocessor dcerpc2: memcap 102400, events [co ] preprocessor dcerpc2_server: default, policy WinXP, \ detect [smb [139,445], tcp 135, udp 135, rpc-over-http-server 593], \ autodetect [tcp 1025:, udp 1025:, rpc-over-http-server 1025:], \ smb_max_chain 3 preprocessor dns: ports { 53 } enable_rdata_overflow preprocessor ssl: ports { 443 465 563 636 989 992 993 994 995 7801 7702 7900 7901 7902 7903 7904 7905 7906 6907 7908 7909 7910 7911 7912 7913 7914 7915 7916 7917 7918 7919 7920 }, trustservers, noinspect_encrypted # SDF sensitive data preprocessor. For more information see README.sensitive_data preprocessor sensitive_data: alert_threshold 25 output alert_full: alert.log output database: log, mysql, user=root password=123456 dbname=snort host=localhost include classification.config include reference.config include $RULE_PATH/local.rules include $RULE_PATH/attack-responses.rules include $RULE_PATH/backdoor.rules include $RULE_PATH/bad-traffic.rules include $RULE_PATH/chat.rules include $RULE_PATH/content-replace.rules include $RULE_PATH/ddos.rules include $RULE_PATH/dns.rules include $RULE_PATH/dos.rules include $RULE_PATH/exploit.rules include $RULE_PATH/finger.rules include $RULE_PATH/ftp.rules include $RULE_PATH/icmp.rules include $RULE_PATH/icmp-info.rules include $RULE_PATH/imap.rules include $RULE_PATH/info.rules include $RULE_PATH/misc.rules include $RULE_PATH/multimedia.rules include $RULE_PATH/mysql.rules include $RULE_PATH/netbios.rules include $RULE_PATH/nntp.rules include $RULE_PATH/oracle.rules include $RULE_PATH/other-ids.rules include $RULE_PATH/p2p.rules include $RULE_PATH/policy.rules include $RULE_PATH/pop2.rules include $RULE_PATH/pop3.rules include $RULE_PATH/rpc.rules include $RULE_PATH/rservices.rules include $RULE_PATH/scada.rules include $RULE_PATH/scan.rules include $RULE_PATH/shellcode.rules include $RULE_PATH/smtp.rules include $RULE_PATH/snmp.rules include $RULE_PATH/specific-threats.rules include $RULE_PATH/spyware-put.rules include $RULE_PATH/sql.rules include $RULE_PATH/telnet.rules include $RULE_PATH/tftp.rules include $RULE_PATH/virus.rules include $RULE_PATH/voip.rules include $RULE_PATH/web-activex.rules include $RULE_PATH/web-attacks.rules include $RULE_PATH/web-cgi.rules include $RULE_PATH/web-client.rules include $RULE_PATH/web-coldfusion.rules include $RULE_PATH/web-frontpage.rules include $RULE_PATH/web-iis.rules include $RULE_PATH/web-misc.rules include $RULE_PATH/web-php.rules include $RULE_PATH/x11.rules include threshold.conf -————————————————————————————- Can anyone help me to solve this problem? Thanks.

Read the article

MonoTouch App Crashes When Returning From MFMailComposeViewController

- by Richard Khan

My MonoTouch Version Info: Release ID: 20401003 Git revision: 2f1746af36f421d262dcd2b0542ce86b12158f02 Build date: 2010-12-23 23:13:38+0000 The MFMailComposeViewController is displayed and works correctly as a dialog using the following code: if (MFMailComposeViewController.CanSendMail) { MFMailComposeViewController mail; mail = new MFMailComposeViewController (); mail.SetSubject ("Subject Test"); mail.SetMessageBody ("Body Test", false); mail.Finished += HandleMailFinished; this.navigationController.PresentModalViewController (mail, true); } else { new UIAlertView ("Mail Failed", "Mail Failed", null, "OK", null).Show (); } However, once the user selects Cancel | Delete Draft or Cancel | Save Draft or Send, the App throws a run-time error like the following: Stacktrace: at (wrapper managed-to-native) MonoTouch.UIKit.UIApplication.UIApplicationMain (int,string[],intptr,intptr) <0x00004 at (wrapper managed-to-native) MonoTouch.UIKit.UIApplication.UIApplicationMain (int,string[],intptr,intptr) <0x00004 at MonoTouch.UIKit.UIApplication.Main (string[],string,string) [0x00038] in /Users/plasma/Source/iphone/monotouch/UIKit/UIApplication.cs:26 at MonoTouch.UIKit.UIApplication.Main (string[]) [0x00000] in /Users/plasma/Source/iphone/monotouch/UIKit/UIApplication.cs:31 at MailDialog.Application.Main (string[]) [0x00000] in /Users/rrkhan/Projects/Sandbox/MailDialog/Main.cs:15 at (wrapper runtime-invoke) .runtime_invoke_void_object (object,intptr,intptr,intptr) Native stacktrace: 0 MailDialog 0x000be66f mono_handle_native_sigsegv + 343 1 MailDialog 0x0000e43e mono_sigsegv_signal_handler + 313 2 libSystem.B.dylib 0x903e946b _sigtramp + 43 3 ??? 0xffffffff 0x0 + 4294967295 4 MessageUI 0x01a9f6b7 -[MFMailComposeController _close] + 284 5 UIKit 0x01f682f1 -[UIActionSheet(Private) _buttonClicked:] + 258 6 UIKit 0x01be1a6e -[UIApplication sendAction:to:from:forEvent:] + 119 7 UIKit 0x01c701b5 -[UIControl sendAction:to:forEvent:] + 67 8 UIKit 0x01c72647 -[UIControl(Internal) _sendActionsForEvents:withEvent:] + 527 9 UIKit 0x01c711f4 -[UIControl touchesEnded:withEvent:] + 458 10 UIKit 0x01c060d1 -[UIWindow _sendTouchesForEvent:] + 567 11 UIKit 0x01be737a -[UIApplication sendEvent:] + 447 12 UIKit 0x01bec732 _UIApplicationHandleEvent + 7576 13 GraphicsServices 0x03eb7a36 PurpleEventCallback + 1550 14 CoreFoundation 0x00df9064 CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE1_PERFORM_FUNCTION + 52 15 CoreFoundation 0x00d596f7 __CFRunLoopDoSource1 + 215 16 CoreFoundation 0x00d56983 __CFRunLoopRun + 979 17 CoreFoundation 0x00d56240 CFRunLoopRunSpecific + 208 18 CoreFoundation 0x00d56161 CFRunLoopRunInMode + 97 19 GraphicsServices 0x03eb6268 GSEventRunModal + 217 20 GraphicsServices 0x03eb632d GSEventRun + 115 21 UIKit 0x01bf042e UIApplicationMain + 1160 22 ??? 0x0a1e4bd9 0x0 + 169757657 23 ??? 0x0a1e4b12 0x0 + 169757458 24 ??? 0x0a1e4515 0x0 + 169755925 25 ??? 0x0a1e4451 0x0 + 169755729 26 ??? 0x0a1e44ac 0x0 + 169755820 27 MailDialog 0x0000e202 mono_jit_runtime_invoke + 1360 28 MailDialog 0x001c92af mono_runtime_invoke + 137 29 MailDialog 0x001caf6b mono_runtime_exec_main + 714 30 MailDialog 0x001ca891 mono_runtime_run_main + 812 31 MailDialog 0x00094fe8 mono_jit_exec + 200 32 MailDialog 0x0027cf05 main + 3494 33 MailDialog 0x00002ca1 _start + 208 34 MailDialog 0x00002bd0 start + 40 Debug info from gdb: warning: Could not find object file "/var/folders/Ny/NyElTwhDGD8kZMqIEeLGXE+++TI/-Tmp-//cc6F1tBs.o" - no debug information available for "template.m". warning: .o file "/Developer/MonoTouch/SDKs/MonoTouch.iphonesimulator4.2.sdk/usr/lib/libmonotouch.a(zlib-helper.x86.42.o)" more recent than executable timestamp in "/Users/rrkhan/Library/Application Support/iPhone Simulator/4.2/Applications/52AF1D24-AADA-48ED-B373-ED08E89E4985/MailDialog.app/MailDialog" warning: Could not open OSO file /Developer/MonoTouch/SDKs/MonoTouch.iphonesimulator4.2.sdk/usr/lib/libmonotouch.a(zlib-helper.x86.42.o) to scan for pubtypes for objfile /Users/rrkhan/Library/Application Support/iPhone Simulator/4.2/Applications/52AF1D24-AADA-48ED-B373-ED08E89E4985/MailDialog.app/MailDialog warning: .o file "/Developer/MonoTouch/SDKs/MonoTouch.iphonesimulator4.2.sdk/usr/lib/libmonotouch.a(monotouch-glue.x86.42.o)" more recent than executable timestamp in "/Users/rrkhan/Library/Application Support/iPhone Simulator/4.2/Applications/52AF1D24-AADA-48ED-B373-ED08E89E4985/MailDialog.app/MailDialog" warning: Could not open OSO file /Developer/MonoTouch/SDKs/MonoTouch.iphonesimulator4.2.sdk/usr/lib/libmonotouch.a(monotouch-glue.x86.42.o) to scan for pubtypes for objfile /Users/rrkhan/Library/Application Support/iPhone Simulator/4.2/Applications/52AF1D24-AADA-48ED-B373-ED08E89E4985/MailDialog.app/MailDialog warning: .o file "/Developer/MonoTouch/SDKs/MonoTouch.iphonesimulator4.2.sdk/usr/lib/libmonotouch.a(gc.x86.42.o)" more recent than executable timestamp in "/Users/rrkhan/Library/Application Support/iPhone Simulator/4.2/Applications/52AF1D24-AADA-48ED-B373-ED08E89E4985/MailDialog.app/MailDialog" warning: Could not open OSO file /Developer/MonoTouch/SDKs/MonoTouch.iphonesimulator4.2.sdk/usr/lib/libmonotouch.a(gc.x86.42.o) to scan for pubtypes for objfile /Users/rrkhan/Library/Application Support/iPhone Simulator/4.2/Applications/52AF1D24-AADA-48ED-B373-ED08E89E4985/MailDialog.app/MailDialog Error connecting stdout and stderr (127.0.0.1:10001) warning: .o file "/Developer/MonoTouch/SDKs/MonoTouch.iphonesimulator4.2.sdk/usr/lib/libmonotouch.a(monotouch-glue.x86.42.o)" more recent than executable timestamp in "/Users/rrkhan/Library/Application Support/iPhone Simulator/4.2/Applications/52AF1D24-AADA-48ED-B373-ED08E89E4985/MailDialog.app/MailDialog" warning: Couldn't open object file '/Developer/MonoTouch/SDKs/MonoTouch.iphonesimulator4.2.sdk/usr/lib/libmonotouch.a(monotouch-glue.x86.42.o)' Attaching to process 9992. Reading symbols for shared libraries . done Reading symbols for shared libraries ....................................................................................................................... done 0x9038e459 in read$UNIX2003 () 8 0x903a8a12 in __workq_kernreturn () 7 "WebThread" 0x903830fa in mach_msg_trap () 6 0x903b10a6 in __semwait_signal () 5 0x90383136 in semaphore_wait_trap () 4 0x903830fa in mach_msg_trap () 3 0x903a8a12 in __workq_kernreturn () 2 "com.apple.libdispatch-manager" 0x903a9982 in kevent () * 1 "com.apple.main-thread" 0x9038e459 in read$UNIX2003 () Thread 8 (process 9992): 0 0x903a8a12 in __workq_kernreturn () 1 0x903a8fa8 in _pthread_wqthread () 2 0x903a8bc6 in start_wqthread () Thread 7 (process 9992): 0 0x903830fa in mach_msg_trap () 1 0x90383867 in mach_msg () 2 0x00df94a6 in __CFRunLoopServiceMachPort () 3 0x00d56874 in __CFRunLoopRun () 4 0x00d56240 in CFRunLoopRunSpecific () 5 0x00d56161 in CFRunLoopRunInMode () 6 0x04f7c423 in RunWebThread () 7 0x903b085d in _pthread_start () 8 0x903b06e2 in thread_start () Thread 6 (process 9992): 0 0x903b10a6 in __semwait_signal () 1 0x903dcee5 in nanosleep$UNIX2003 () 2 0x903dce23 in usleep$UNIX2003 () 3 0x0027714c in monotouch_pump_gc () 4 0x903b085d in _pthread_start () 5 0x903b06e2 in thread_start () Thread 5 (process 9992): 0 0x90383136 in semaphore_wait_trap () 1 0x0015ae1d in finalizer_thread (unused=0x0) at ../../../../mono/metadata/gc.c:1026 2 0x002034a3 in start_wrapper (data=0x7b16ba0) at ../../../../mono/metadata/threads.c:661 3 0x002448e2 in thread_start_routine (args=0x8037e34) at ../../../../mono/io-layer/wthreads.c:286 4 0x00274357 in GC_start_routine (arg=0x6ff7f60) at ../../../libgc/pthread_support.c:1390 5 0x903b085d in _pthread_start () 6 0x903b06e2 in thread_start () Thread 4 (process 9992): 0 0x903830fa in mach_msg_trap () 1 0x90383867 in mach_msg () 2 0x0011cc46 in mach_exception_thread (arg=0x0) at ../../../../mono/mini/mini-darwin.c:138 3 0x903b085d in _pthread_start () 4 0x903b06e2 in thread_start () Thread 3 (process 9992): 0 0x903a8a12 in __workq_kernreturn () 1 0x903a8fa8 in _pthread_wqthread () 2 0x903a8bc6 in start_wqthread () Thread 2 (process 9992): 0 0x903a9982 in kevent () 1 0x903aa09c in _dispatch_mgr_invoke () 2 0x903a9559 in _dispatch_queue_invoke () 3 0x903a92fe in _dispatch_worker_thread2 () 4 0x903a8d81 in _pthread_wqthread () 5 0x903a8bc6 in start_wqthread () Thread 1 (process 9992): 0 0x9038e459 in read$UNIX2003 () 1 0x000be81f in mono_handle_native_sigsegv (signal=11, ctx=0xbfffd238) at ../../../../mono/mini/mini-exceptions.c:1826 2 0x0000e43e in mono_sigsegv_signal_handler (_dummy=10, info=0xbfffd1f8, context=0xbfffd238) at ../../../../mono/mini/mini.c:4846 3 4 0x028d6a63 in objc_msgSend () 5 0x01ad469f in func.24012 () 6 0x01a9f6b7 in -[MFMailComposeController _close] () 7 0x01f682f1 in -[UIActionSheet(Private) _buttonClicked:] () 8 0x01be1a6e in -[UIApplication sendAction:to:from:forEvent:] () 9 0x01c701b5 in -[UIControl sendAction:to:forEvent:] () 10 0x01c72647 in -[UIControl(Internal) _sendActionsForEvents:withEvent:] () 11 0x01c711f4 in -[UIControl touchesEnded:withEvent:] () 12 0x01c060d1 in -[UIWindow _sendTouchesForEvent:] () 13 0x01be737a in -[UIApplication sendEvent:] () 14 0x01bec732 in _UIApplicationHandleEvent () 15 0x03eb7a36 in PurpleEventCallback () 16 0x00df9064 in CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE1_PERFORM_FUNCTION () 17 0x00d596f7 in __CFRunLoopDoSource1 () 18 0x00d56983 in __CFRunLoopRun () 19 0x00d56240 in CFRunLoopRunSpecific () 20 0x00d56161 in CFRunLoopRunInMode () 21 0x03eb6268 in GSEventRunModal () 22 0x03eb632d in GSEventRun () 23 0x01bf042e in UIApplicationMain () 24 0x0a1e4bd9 in ?? () 25 0x0a1e4b12 in ?? () 26 0x0a1e4515 in ?? () 27 0x0a1e4451 in ?? () 28 0x0a1e44ac in ?? () 29 0x0000e202 in mono_jit_runtime_invoke (method=0xa806e6c, obj=0x0, params=0xbfffedbc, exc=0x0) at ../../../../mono/mini/mini.c:4733 30 0x001c92af in mono_runtime_invoke (method=0xa806e6c, obj=0x0, params=0xbfffedbc, exc=0x0) at ../../../../mono/metadata/object.c:2615 31 0x001caf6b in mono_runtime_exec_main (method=0xa806e6c, args=0xa6a34e0, exc=0x0) at ../../../../mono/metadata/object.c:3581 32 0x001ca891 in mono_runtime_run_main (method=0xa806e6c, argc=0, argv=0xbfffeef4, exc=0x0) at ../../../../mono/metadata/object.c:3355 33 0x00094fe8 in mono_jit_exec (domain=0x6f8fe58, assembly=0xa200730, argc=1, argv=0xbfffeef0) at ../../../../mono/mini/driver.c:1094 34 0x0027cf05 in main () ================================================================= Got a SIGSEGV while executing native code. This usually indicates a fatal error in the mono runtime or one of the native libraries used by your application. Unhandled Exception: System.NullReferenceException: Object reference not set to an instance of an object at (wrapper managed-to-native) MonoTouch.UIKit.UIApplication:UIApplicationMain (int,string[],intptr,intptr) at MonoTouch.UIKit.UIApplication.Main (System.String[] args, System.String principalClassName, System.String delegateClassName) [0x00038] in /Users/plasma/Source/iphone/monotouch/UIKit/UIApplication.cs:26 at MonoTouch.UIKit.UIApplication.Main (System.String[] args) [0x00000] in /Users/plasma/Source/iphone/monotouch/UIKit/UIApplication.cs:31 at MailDialog.Application.Main (System.String[] args) [0x00000] in /Users/rrkhan/Projects/Sandbox/MailDialog/Main.cs:15 I have a very simple sample project illustrating the problem. I can sent you if required.

Read the article

embedded glassfish: java.lang.NoClassDefFoundError: java/util/ServiceLoader

- by Xinus

I am trying to embed glassfish inside my java program using embeded api, I am using maven2 and its pom.xml is as follows <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>orh.highmark</groupId> <artifactId>glassfish-test1</artifactId> <version>1.0-SNAPSHOT</version> <packaging>jar</packaging> <name>glassfish-test1</name> <url>http://maven.apache.org</url> <properties> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> </properties> <dependencies> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>3.8.1</version> <scope>test</scope> </dependency> <dependency> <groupId>org.glassfish.extras</groupId> <artifactId>glassfish-embedded-all</artifactId> <version>3.1-SNAPSHOT</version> </dependency> </dependencies> <repositories> <repository> <id>maven2-repository.dev.java.net</id> <name>Java.net Repository for Maven</name> <url>http://download.java.net/maven/2/</url> <layout>default</layout> </repository> <repository> <id>glassfish-repository</id> <name>GlassFish Nexus Repository</name> <url>http://maven.glassfish.org/content/groups/glassfish</url> </repository> </repositories> <build> <plugins> <plugin> <groupId>org.codehaus.mojo</groupId> <artifactId>exec-maven-plugin</artifactId> <version>1.1</version> <executions> <execution> <goals> <goal>java</goal> </goals> </execution> </executions> <configuration> <mainClass>orh.highmark.App</mainClass> <arguments> <argument>argument1</argument> </arguments> </configuration> </plugin> </plugins> </build> </project> Program: public class App { public static void main( String[] args ) { System.out.println( "Hello World!" ); Server.Builder builder = new Server.Builder("test"); builder.logger(true); Server server = builder.build(); } } But for some reason it always gives me error as java.lang.NoClassDefFoundError: java/util/ServiceLoader here is the output C:\Users\sunils\glassfish-tests\glassfish-test1>mvn -e exec:java + Error stacktraces are turned on. [INFO] Scanning for projects... [INFO] ------------------------------------------------------------------------ [INFO] Building glassfish-test1 [INFO] task-segment: [exec:java] [INFO] ------------------------------------------------------------------------ [INFO] Preparing exec:java [INFO] No goals needed for project - skipping [INFO] [exec:java {execution: default-cli}] Hello World! [INFO] ------------------------------------------------------------------------ [ERROR] BUILD ERROR [INFO] ------------------------------------------------------------------------ [INFO] An exception occured while executing the Java class. null java/util/ServiceLoader [INFO] ------------------------------------------------------------------------ [INFO] Trace org.apache.maven.lifecycle.LifecycleExecutionException: An exception occured whi le executing the Java class. null at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(Defa ultLifecycleExecutor.java:719) at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeStandalone Goal(DefaultLifecycleExecutor.java:569) at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(Defau ltLifecycleExecutor.java:539) at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHan dleFailures(DefaultLifecycleExecutor.java:387) at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegmen ts(DefaultLifecycleExecutor.java:348) at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLi fecycleExecutor.java:180) at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:328) at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:138) at org.apache.maven.cli.MavenCli.main(MavenCli.java:362) at org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:6 0) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl. java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces sorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315) at org.codehaus.classworlds.Launcher.launch(Launcher.java:255) at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430) at org.codehaus.classworlds.Launcher.main(Launcher.java:375) Caused by: org.apache.maven.plugin.MojoExecutionException: An exception occured while executing the Java class. null at org.codehaus.mojo.exec.ExecJavaMojo.execute(ExecJavaMojo.java:345) at org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPlugi nManager.java:490) at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(Defa ultLifecycleExecutor.java:694) ... 17 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl. java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces sorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:290) at java.lang.Thread.run(Thread.java:595) Caused by: java.lang.NoClassDefFoundError: java/util/ServiceLoader at org.glassfish.api.embedded.Server.getMain(Server.java:701) at org.glassfish.api.embedded.Server.<init>(Server.java:290) at org.glassfish.api.embedded.Server.<init>(Server.java:75) at org.glassfish.api.embedded.Server$Builder.build(Server.java:185) at org.glassfish.api.embedded.Server$Builder.build(Server.java:167) at orh.highmark.App.main(App.java:14) ... 6 more [INFO] ------------------------------------------------------------------------ [INFO] Total time: 1 second [INFO] Finished at: Sat May 08 02:55:03 IST 2010 [INFO] Final Memory: 3M/6M [INFO] ------------------------------------------------------------------------ I couldn't guess its problem with my program or with the glassfish api. Can somebody please help me understand what is happening here and how to rectify it? thanks for any clue ..

Read the article

"Exception has been thrown by the target of an invocation" Running Tests - VS2008 SP1

- by omatrot

I'm using Visual Studio 2008 Team Suite and I'm unable to run tests and display the Test/Windows/Test Result Window. The result is a dialog box with the following content : "Exception has been thrown by the target of an invocation". Team Explorer has been installed after Visual Studio 2008 SP1. So I have re-apllied the service pack. Searching the web I found that this error is pretty common but unfortunately, the proposed solutions does not work for me. The problem was never analysed so I decided to give it a try : I reproduced the problem on a computer, attached the process with windbg and start with the basic investigations. Following are the first results : 0:000>!dumpstack OS Thread Id: 0xdb0 (0) Current frame: USER32!NtUserWaitMessage+0x15 ChildEBP RetAddr Caller,Callee 003fec94 75a32674 USER32!DialogBox2+0x222, calling USER32!NtUserWaitMessage 003fecd0 75a3288a USER32!InternalDialogBox+0xe5, calling USER32!DialogBox2 003fecfc 75a6f8d0 USER32!SoftModalMessageBox+0x757, calling USER32!InternalDialogBox 003fed3c 6eb61996 mscorwks!Thread::ReverseLeaveRuntime+0x95, calling mscorwks!_EH_epilog3 003fedb0 75a6fbac USER32!MessageBoxWorker+0x269, calling USER32!SoftModalMessageBox 003fede0 6ea559c3 mscorwks!SetupThreadNoThrow+0x19a, calling mscorwks!_EH_epilog3_catch_GS 003fee24 6eb61d8a mscorwks!HasIllegalReentrancy+0xac, calling mscorwks!_EH_epilog3 003fee30 6ea89796 mscorwks!SimpleComCallWrapper::Release+0x2e, calling mscorwks!CompareExchangeMP 003fee38 6ea0da05 mscorwks!CLRException::HandlerState::CleanupTry+0x16, calling mscorwks!GetCurrentSEHRecord 003fee44 6ea0c9c0 mscorwks!Thread::EnablePreemptiveGC+0xf, calling mscorwks!Thread::CatchAtSafePoint 003fee4c 6ea8a241 mscorwks!Unknown_Release_Internal+0x24d, calling mscorwks!GCHolder<1,0,0>::Pop 003fee50 6ea0c86c mscorwks!_EH_epilog3_catch_GS+0xa, calling mscorwks!__security_check_cookie 003fee54 6ea8a24c mscorwks!Unknown_Release_Internal+0x258, calling mscorwks!_EH_epilog3_catch_GS 003fee7c 75a16941 USER32!UserCallWinProcCheckWow+0x13d, calling ntdll!RtlDeactivateActivationContextUnsafeFast 003feed8 7082119e msenv!ATL::CComCritSecLock<ATL::CComCriticalSection>::Lock+0xd, calling ntdll!RtlEnterCriticalSection 003fef08 75a6fe5b USER32!MessageBoxIndirectW+0x2e, calling USER32!MessageBoxWorker 003fef7c 70a1e367 msenv!MessageBoxPVoidW+0xda 003fefd4 70a1db60 msenv!VBDialogCover2+0x11b 003ff01c 70a1e4c0 msenv!VBMessageBox2W+0xf0, calling msenv!VBDialogCover2 003ff044 7087246b msenv!main_GetAppNameW+0xa, calling msenv!GetAppNameInternal 003ff04c 70a1e4f2 msenv!VBMessageBox3W+0x1c, calling msenv!VBMessageBox2W 003ff064 70a1d6d7 msenv!_IdMsgShow+0x362, calling msenv!VBMessageBox3W 003ff0cc 70951841 msenv!TaskDialogCallback+0x7e0, calling msenv!_IdMsgShow 003ff118 6eb20da4 mscorwks!Unknown_QueryInterface+0x230, calling mscorwks!_EH_epilog3_catch_GS 003ff14c 6eb20c43 mscorwks!Unknown_QueryInterface_Internal+0x3d8, calling mscorwks!_EH_epilog3_catch_GS 003ff168 02006ec4 02006ec4, calling 0247a1e8 003ff16c 6ea0c86c mscorwks!_EH_epilog3_catch_GS+0xa, calling mscorwks!__security_check_cookie 003ff198 6eb20562 mscorwks!COMToCLRWorker+0xb34, calling mscorwks!_EH_epilog3_catch_GS 003ff19c 0247a235 0247a235, calling mscorwks!COMToCLRWorker 003ff1c4 7083249f msenv!CVSCommandTarget::ExecCmd+0x937 003ff1e4 7086d5c8 msenv!VsReportErrorInfo+0x11, calling msenv!TaskDialogCallback+0xd8 003ff1f8 7093e65b msenv!CVSCommandTarget::ExecCmd+0x945, calling msenv!VsReportErrorInfo 003ff25c 7081f53a msenv!ATL::CComPtr<IVsLanguageInfo>::~CComPtr<IVsLanguageInfo>+0x24, calling msenv!_EH_epilog3 003ff260 70b18d72 msenv!LogCommand+0x4c, calling msenv!ATL::CComPtr<IVsCodePageSelection>::~CComPtr<IVsCodePageSelection> 003ff264 70b18d77 msenv!LogCommand+0x51, calling msenv!_EH_epilog3 003ff280 70a4fd0e msenv!CMsoButtonUser::FClick+0x1d1, calling msenv!CVSCommandTarget::ExecCmd 003ff2f4 70823a87 msenv!CTLSITE::QueryInterface+0x16 003ff31c 70cb7d4d msenv!TBCB::FNotifyFocus+0x204 003ff35c 70ce5fda msenv!TB::NotifyControl+0x101 003ff3bc 709910f6 msenv!TB::FRequestFocus+0x4ed, calling msenv!TB::NotifyControl 003ff414 708254ba msenv!CMsoButtonUser::FEnabled+0x3d, calling msenv!GetQueryStatusFlags 003ff428 7086222a msenv!TBC::FAutoEnabled+0x24 003ff43c 7098e1eb msenv!TB::LProcessInputMsg+0xdb4 003ff458 6bec1c49 (MethodDesc 0x6bcd7f54 +0x89 System.Windows.Forms.Form.DefWndProc(System.Windows.Forms.Message ByRef)), calling 6be3b738 003ff50c 70823ab0 msenv!FPtbFromSite+0x16 003ff520 70991c43 msenv!TB::PtbParent+0x25, calling msenv!FPtbFromSite 003ff52c 708dda49 msenv!TBWndProc+0x2da 003ff588 0203d770 0203d770, calling 0247a1e8 003ff598 70822a70 msenv!CPaneFrame::Release+0x118, calling msenv!_EH_epilog3 003ff5b0 75a16238 USER32!InternalCallWinProc+0x23 003ff5dc 75a168ea USER32!UserCallWinProcCheckWow+0x109, calling USER32!InternalCallWinProc 003ff620 75a16899 USER32!UserCallWinProcCheckWow+0x6a, calling ntdll!RtlActivateActivationContextUnsafeFast 003ff654 75a17d31 USER32!DispatchMessageWorker+0x3bc, calling USER32!UserCallWinProcCheckWow 003ff688 70847f2b msenv!CMsoComponent::FPreTranslateMessage+0x72, calling msenv!MainFTranslateMessage 003ff6b4 75a17dfa USER32!DispatchMessageW+0xf, calling USER32!DispatchMessageWorker 003ff6c4 70831553 msenv!EnvironmentMsgLoop+0x1ea, calling USER32!DispatchMessageW 003ff6f8 708eb9bd msenv!CMsoCMHandler::FPushMessageLoop+0x86, calling msenv!EnvironmentMsgLoop 003ff724 708eb94d msenv!SCM::FPushMessageLoop+0xb7 003ff74c 708eb8e9 msenv!SCM_MsoCompMgr::FPushMessageLoop+0x28, calling msenv!SCM::FPushMessageLoop 003ff768 708eb8b8 msenv!CMsoComponent::PushMsgLoop+0x28 003ff788 708ebe4e msenv!VStudioMainLogged+0x482, calling msenv!CMsoComponent::PushMsgLoop 003ff7ac 70882afe msenv!CVsActivityLogSingleton::Instance+0xdf, calling msenv!_EH_epilog3 003ff7d8 70882afe msenv!CVsActivityLogSingleton::Instance+0xdf, calling msenv!_EH_epilog3 003ff7dc 707e4e31 msenv!VActivityLogStartupEntries+0x42 003ff7f4 7081f63b msenv!ATL::CComPtr<IClassFactory>::~CComPtr<IClassFactory>+0x24, calling msenv!_EH_epilog3 003ff7f8 708b250f msenv!ATL::CComQIPtr<IUnknown,&IID_IUnknown>::~CComQIPtr<IUnknown,&IID_IUnknown>+0x1d, calling msenv!_EH_epilog3 003ff820 708e7561 msenv!VStudioMain+0xc1, calling msenv!VStudioMainLogged 003ff84c 2f32aabc devenv!util_CallVsMain+0xff 003ff878 2f3278f2 devenv!CDevEnvAppId::Run+0x11fd, calling devenv!util_CallVsMain 003ff97c 77533b23 ntdll!RtlpAllocateHeap+0xe73, calling ntdll!_SEH_epilog4 003ff9f0 77536cd7 ntdll!RtlpLowFragHeapAllocFromContext+0x882, calling ntdll!RtlpSubSegmentInitialize 003ffa10 7753609f ntdll!RtlNtStatusToDosError+0x3b, calling ntdll!RtlNtStatusToDosErrorNoTeb 003ffa14 775360a4 ntdll!RtlNtStatusToDosError+0x40, calling ntdll!_SEH_epilog4 003ffa40 775360a4 ntdll!RtlNtStatusToDosError+0x40, calling ntdll!_SEH_epilog4 003ffa44 75bd2736 kernel32!LocalBaseRegOpenKey+0x159, calling ntdll!RtlNtStatusToDosError 003ffa48 75bd2762 kernel32!LocalBaseRegOpenKey+0x22a, calling kernel32!_SEH_epilog4 003ffac4 75bd2762 kernel32!LocalBaseRegOpenKey+0x22a, calling kernel32!_SEH_epilog4 003ffac8 75bd28c9 kernel32!RegOpenKeyExInternalW+0x130, calling kernel32!LocalBaseRegOpenKey 003ffad8 75bd28de kernel32!RegOpenKeyExInternalW+0x211 003ffae0 75bd28e5 kernel32!RegOpenKeyExInternalW+0x21d, calling kernel32!_SEH_epilog4 003ffb04 6f282e2b MSVCR90!_unlock+0x15, calling ntdll!RtlLeaveCriticalSection 003ffb14 75bd2642 kernel32!BaseRegCloseKeyInternal+0x41, calling ntdll!NtClose 003ffb28 75bd25d0 kernel32!RegCloseKey+0xd4, calling kernel32!_SEH_epilog4 003ffb5c 75bd25d0 kernel32!RegCloseKey+0xd4, calling kernel32!_SEH_epilog4 003ffb60 2f321ea4 devenv!DwInitSyncObjects+0x340 003ffb90 2f327bf4 devenv!WinMain+0x74, calling devenv!CDevEnvAppId::Run 003ffbac 2f327c68 devenv!License::GetPID+0x258, calling devenv!WinMain 003ffc3c 75bd3677 kernel32!BaseThreadInitThunk+0xe 003ffc48 77539d72 ntdll!__RtlUserThreadStart+0x70 003ffc88 77539d45 ntdll!_RtlUserThreadStart+0x1b, calling ntdll!__RtlUserThreadStart 0:000> !pe -nested Exception object: 050aae9c Exception type: System.Reflection.TargetInvocationException Message: Exception has been thrown by the target of an invocation. InnerException: System.NullReferenceException, use !PrintException 050aac64 to see more StackTrace (generated): SP IP Function 003FEC2C 6D2700F7 mscorlib_ni!System.RuntimeType.CreateInstanceSlow(Boolean, Boolean)+0x57 003FEC5C 6D270067 mscorlib_ni!System.RuntimeType.CreateInstanceImpl(Boolean, Boolean, Boolean)+0xe7 003FEC94 6D270264 mscorlib_ni!System.Activator.CreateInstance(System.Type, Boolean)+0x44 003FECA4 6AD02DAF Microsoft_VisualStudio_Shell_9_0_ni!Microsoft.VisualStudio.Shell.Package.CreateToolWindow(System.Type, Int32, Microsoft.VisualStudio.Shell.ProvideToolWindowAttribute)+0x67 003FED30 6AD0311B Microsoft_VisualStudio_Shell_9_0_ni!Microsoft.VisualStudio.Shell.Package.CreateToolWindow(System.Type, Int32)+0xb7 003FED58 6AD02D12 Microsoft_VisualStudio_Shell_9_0_ni!Microsoft.VisualStudio.Shell.Package.FindToolWindow(System.Type, Int32, Boolean, Microsoft.VisualStudio.Shell.ProvideToolWindowAttribute)+0x7a 003FED88 6AD02D39 Microsoft_VisualStudio_Shell_9_0_ni!Microsoft.VisualStudio.Shell.Package.FindToolWindow(System.Type, Int32, Boolean)+0x11 003FED94 02585E30 Microsoft_VisualStudio_QualityTools_TestCaseManagement!Microsoft.VisualStudio.TestTools.TestCaseManagement.QualityToolsPackage.InitToolWindowVariable[[System.__Canon, mscorlib]](System.__Canon ByRef, System.String, Boolean)+0x58 003FEDD0 02585DBE Microsoft_VisualStudio_QualityTools_TestCaseManagement!Microsoft.VisualStudio.TestTools.TestCaseManagement.QualityToolsPackage.InitToolWindowVariable[[System.__Canon, mscorlib]](System.__Canon ByRef, System.String)+0x36 003FEDE4 02585D32 Microsoft_VisualStudio_QualityTools_TestCaseManagement!Microsoft.VisualStudio.TestTools.TestCaseManagement.QualityToolsPackage.ShowToolWindow[[System.__Canon, mscorlib]](System.__Canon ByRef, System.String, Boolean)+0x3a 003FEE00 02585AB4 Microsoft_VisualStudio_QualityTools_TestCaseManagement!Microsoft.VisualStudio.TestTools.TestCaseManagement.QualityToolsPackage.OpenTestResultsToolWindow()+0x2c 003FEE10 02585A6E Microsoft_VisualStudio_QualityTools_TestCaseManagement!Microsoft.VisualStudio.TestTools.TestCaseManagement.QualityToolsPackage.OnMenuViewTestResults(System.Object, System.EventArgs)+0x6 003FEE18 6CD4F993 System_ni!System.ComponentModel.Design.MenuCommand.Invoke()+0x43 003FEE40 6CD4F9D4 System_ni!System.ComponentModel.Design.MenuCommand.Invoke(System.Object)+0x8 003FEE48 6AD000FA Microsoft_VisualStudio_Shell_9_0_ni!Microsoft.VisualStudio.Shell.OleMenuCommandService.Microsoft.VisualStudio.OLE.Interop.IOleCommandTarget.Exec(System.Guid ByRef, UInt32, UInt32, IntPtr, IntPtr)+0x11a 003FEEA0 6AD03FB8 Microsoft_VisualStudio_Shell_9_0_ni!Microsoft.VisualStudio.Shell.Package.Microsoft.VisualStudio.OLE.Interop.IOleCommandTarget.Exec(System.Guid ByRef, UInt32, UInt32, IntPtr, IntPtr)+0x44 StackTraceString: <none> HResult: 80131604 0:000> !PrintException 050aac64 Exception object: 050aac64 Exception type: System.NullReferenceException Message: Object reference not set to an instance of an object. InnerException: <none> StackTrace (generated): SP IP Function 003FE660 078E60BE Microsoft_VisualStudio_TeamSystem_Integration!Microsoft.VisualStudio.TeamSystem.Integration.TcmResultsPublishManager..ctor(Microsoft.VisualStudio.TeamSystem.Integration.ResultsPublishManager)+0xc6 003FE674 078E5C91 Microsoft_VisualStudio_TeamSystem_Integration!Microsoft.VisualStudio.TeamSystem.Integration.ResultsPublishManager..ctor(Microsoft.VisualStudio.TeamSystem.Integration.TeamFoundationHostHelper)+0x59 003FE684 078E2FA0 Microsoft_VisualStudio_TeamSystem_Integration!Microsoft.VisualStudio.TeamSystem.Integration.VsetServerHelper..ctor(System.IServiceProvider)+0x50 003FE6A4 078E2E90 Microsoft_VisualStudio_TeamSystem_Common!Microsoft.VisualStudio.TeamSystem.Integration.Client.VsetHelper.InitializeThrow(System.IServiceProvider)+0x20 003FE6B8 078E2E2A Microsoft_VisualStudio_TeamSystem_Common!Microsoft.VisualStudio.TeamSystem.Integration.Client.VsetHelper.InitializeHelper(System.IServiceProvider)+0x22 003FE6E0 078E2DEC Microsoft_VisualStudio_TeamSystem_Common!Microsoft.VisualStudio.TeamSystem.Integration.Client.VsetHelper.CreateVsetHelper(System.IServiceProvider)+0x1c 003FE6F0 078E2DAC Microsoft_VisualStudio_QualityTools_TestCaseManagement!Microsoft.VisualStudio.TestTools.TestCaseManagement.QualityToolsPackage.get_VsetHelper()+0x14 003FE6F8 02586BBE Microsoft_VisualStudio_QualityTools_TestCaseManagement!Microsoft.VisualStudio.TestTools.TestCaseManagement.ResultsToolWindow..ctor()+0x9f6 003FE798 02585F8A Microsoft_VisualStudio_QualityTools_TestCaseManagement!Microsoft.VisualStudio.TestTools.TestCaseManagement.ResultToolWindowHost..ctor()+0x1a StackTraceString: <none> HResult: 80004003 In order to be able to continue the analysis, we need to get the parameters to see what is going on. I also tried to run devenv.exe with the /log switch. No error in the log after reproducing the problem. Finally, If Team Explorer is removed from the system, the problem goes away. Any help appreciated. TIA. Olivier.

Search Results

Search found 25386 results on 1016 pages for 'zend test'.

Page 816/1016 | < Previous Page | 812 813 814 815 816 817 818 819 820 821 822 823 | Next Page >

- by Derek Pressnall

- by Don

- by user1717735

- by Alex

- by shadowfoxmi

- by Arno2501

- by Ciaran87Bel

- by balupton

- by Chan

- by Shaun

- by ScottGu

- by Onno

- by user12620111

- by Sergey Litvinov

- by Justin

- by Eli Courtwright

- by aspiehler

- by Niko

- by drachenstern

- by Marcos Carceles

- by AdRock

- by manofseven2

- by Richard Khan

- by Xinus

- by omatrot

< Previous Page | 812 813 814 815 816 817 818 819 820 821 822 823 | Next Page >