• EU General Data Protection Regulation (GDPR) and Data Protection Officer (DPO) as well as lot of discussion and presentations about Network Information Security (NIS).
• C-Lion1 is being branched to Hanko, Finland.
  
• Constant monitoring of security issues is required with automated alerts. Common configuration mistakes are repeated over and over again. Without automated monitoring and alerts, situation would be really bad. It's not good with monitoring, but it's much better than it would be without it.
  
• Again kicked a few new integrations into production. Integration isn't that hard, when you just got a great process and experience and flexibility in implementing it. Often the slowest part of integration is using some 'great generic extensive suits for everything standard'. Which actually makes implementation very slow and hard compared to building what's required using modern tools.
  
• Just noticed that StartSSL / StartCom- Website has been overhauled. Nice. I were looking for Organization Validated (OV) SSL certs. It's also really nice that they provide free EV SSL certs. They also provide everything you need to encrypt your confidential emails using S/MIME. There were also some news about StartCom signing SSL certificates for existing domains and so on. Which is yet another reason why current CA system is just so broken.
  
• Implementing your own recommender systems in Python - Nice article. Been there, done that.
  
• Enjoyed some semi-serious storage subsystem issues with one hosting provider. Only good thing about this is knowing that competent people are working on this already and there's no need to go ballistic and figure out some hard to troubleshoot issues which you don't know enough. - Phew. That's good, bad and the ugly.
  
• Once again encountered newbie / hopeless process guys. They were totally surprised that one thing didn't work. Yeah, it works, when it works, when it doesn't you should follow your fallback / secondary process which you practice frequently. - Follow what? Nothing works, panic panic. Well, obviously something works, you called me, you sent me email. So how do you say nothing works? - What we should learn from this story? There's just so many people who are clueless and trust things which "always work". - But that's exactly the source problem. You shouldn't think that things always work. You could lose water, you could lose power, you could lose heating (that's bad in Finland, it could be -35C or worse). But if you've done sane plans, you should just follow well rehearsed backup procedures. What is the problem?  - This reminds me still from the Data Center guys which said it was a nightmare and huge problem that DC lost power for a few seconds. Sure, it did. So what? Anything actually important shouldn't be dependent on one data center or actually the so called 'The Internet' being available. - If you trust something vital to these things, then YOU'RE the problem. The failure wasn't the actual problem, yeah it was inconvenient but so what?
  
• It remains to be seen what kind of nightmares Internet of Things will cause. When will people start dying when mobile data didn't work. Well, your cloud based heart pacemaker didn't work because loss of Internet connectivity. Well, there you go. You're happy that only a few people died. Let's see when planes and cruise ships and stuff like that starts crashing and sinking when "The Internet was lost". Sigh. Yes, Finnish Coast Guard just warned about these guys, who think that the only way to navigate a ship is the mobile app on Android phone which loads stuff in real-time from Internet and uses GPS. When any of those things fail, good luck. You're going to have a bad time and you're totally lost at sea. - I think actually that it's pretty much time for Darwin awards. You were asking for it, and you got what you wanted and what's best for human kind and the universe.
  
• Sometimes these people make me to think about fun mind games. What if just for fun, we would let the GPS navigation system to drift 20 meters in one hour globally to random direction. And then return it back. It really shouldn't cause any real problems? It would be just way too fun to see what kind of problems that would cause. Yes, this is all because there 'we trust this technology fully' people mentioned above. Nobody should trust GPS for anything important or which could cause damage or worse.
  
• It's also important to remember that you're usually getting what you're paying for. Are you paying for real-time replication to another continent? Are you paying for your own national spectrum to delivery our content over dedicated radio network if Internet fails, etc. Some customers, are, most aren't. Does your employees have satellite phones and aggregates with enough fuel? This just in case to reach them if there's national extended power black out. Ok, they don't? What's the plan to contact them in that case? Etc.
  

• Progress Towards 100% HTTPS, June 2016 - Let's Encrypt - Free SSL/TLS Certificates - Very nice. It would be great to get all sites to use HTTP/2 with HTTPS of course aka H2.
• More quality software. Skype on Linux is losing half of text being written and so on. So much quality software. Pure love. I wonder what's wrong with the keyboard input handler.
  
• It seems that TP-LINK TD-W9980 V1 firmware TD-W9980_V1_160125 is still buggy. It keeps losing IPv6 configuration. Fixing it? I'll be happy to provide more details. World is just so full of quality software. Once again. Temporary workaround? One server in network checks if IPv6 is available, if it isn't. It connects the modem using script and re-enables it. Sure it works, but it's totally brain dead way of dealing with this issue. (I know, this isn't going to be first nor last workaround I had to implement for others crappy code, but that's just life.) I just need to make this stuff work.
• Not enough bad software yet? Just found out that the web shop where I wanted to order some stuff didn't work properly and I couldn't pay the purchases. I've been wondering how much money they have to make, to be so rich they're ignoring paying customers and tell them to get lost. Keep your money, we've got so much we don't know what we would do with it anyway. - Thank you for rejecting me. - Yet, I found another supplier of course. They accepted my money.
  
• Re-checked Sigfox integration API and Backend RESTful connectors with JSON to confirm that those are suitable for the project requirements. Also the messaging limits are pretty important 12 bytes and 140 msg/s day. Communication can be initiated only by the device, not by the network.
  
• Configured a new 8 TB archive storage drive for backups / long term low demand data storage, used smartctl to run extended self-test, then run badblocks to run proper read write media tests and tune2fs to configure fcsk boot options, fstab to set writeback mode for ramdisk like write performance for small non fsynced writes. hdparm to set preferred power saving options.
• Disabling paste on password fields - Nice post. Disabling paste will guarantee bad passwords.
  
• A nice analysis about Apple's new Apple File System (APFS). kw: NAND flash-aware characteristics, blocks,pages, FTL, SSD, ECC, ZFS, metadata, checksumming, metadata and data consistency, HFS, HFS+.
• I've been using NTFS on USB Flash Memory Sticks and external drivers for over 5 years when using Linux / Windows mixed use. No problems so far. This of course wouldn't mean it's problem free or 'rock solid', but in normal daily use I haven't encountered any. Why? Because meta data journaling. exFAT works as well, but it doesn't journal -> less reliable. - This is just an answer to people asking if NTFS works with Linux, it does.
  
• Kerv is cool, some say. I guess there's rate limits. So there's single payment limit as well as daily payment limit + notification / automatic suspension, if something is "out of the ordinary". - Yet in technical terms, nothing new. I just can't stand this all "this is so bleep" with something new, which isn't technically anything new. Actually technically something really awesome and new happens extremely rarely. I guess Kerv is basically yet another implementation of the "NFC payment stickers" which one operator in Finland has been pushing out as mobile payments. You just need to glue that sticker on your phone. What kind of stupid marketing lies are those.
  
• Rechecked Apache Kafka - messaging system. - No I don't see use for it right now. But might become handy with some project.
• It seems that LoadAverage.org is finally back online.

• After I switched email hosting I found out what I were expecting and suspecting earlier, but I hadn't had it confirmed. LinkedIn uses email batching, delivering emails in large batches to individual servers / domains. Now when I use different domain, I just get a ripple of the email. When I used my own domain and server, I got everything at once. Because it's just more efficient to deliver emails at once, instead of delivering those in multiple small batches.
• It's also unsurprising that new email hosting is much slower than it used to be. I were used to everything happening immediately but now there's latencies related to everything. Previous server got under 1 ms latency and basically everything in RAM memory. Now data is being fetched over much slower link and of course it's not all in RAM. It's really easy to notice sluggishness on every turn when using the new service. With the old server worst case was that data wasn't in RAM and it had to be read from SSD SAN.
• Oscobo replied to me linking a few articles. But none of those answer to the fundamental questions I'm asking. They haven't given out any information what information is being forwarded to 3rd parties like Microsoft Bing to fetch the actual results. It would be really nice to know. Afaik, this is pretty important question. It's easy to claim that things are secure in a way, that it looks good, but in reality we all know that it might still be very insecure, either by accident or incompetence or by intentional design.
• Something different: Read multiple articles on The UAAV Digest. It seems that military drones are developing really fast, much faster (no surprise) than consumer ones. It's very interesting to see if (when) drones can practically replace hugely expensive air craft carriers. kw: TERN, VTOL, DARPA
• Some people warn that Telegram isn't encrypted. Sure it's encrypted, but ... It's not End to End Encrypted (E2EE) unless Secret Mode is being used. Many of the Telegram clients don't even support E2EE encrypted Telegram Secret chat mode. Data is still delivered over encrypted connection, but it's not end2end encrypted. If you've got something truly confidential, who would use any "secure app" anyway? I'm sure the stake holders wouldn't love such actions and it would be blatant breach of confidentiality.
• I guess the new Russian data collection law isn't that bad? At least they're openly admitting it. Other countries might do exactly the same, or worse. They're just not telling about it. Basically all communications need to be stored for six months in plaintext and if encrypted there needs to be a backdoor for deciphering the communications.
• Ordered a set of programmable NXP NFC tags - just for fun, play and experiment. Let's see if I can figure any practical use for those. Maybe configuring visitor WLAN at office could be one use.
• Checked out Bluetooth 5. 4x the range and 2x the bandwidth. Whats even better then Bluetooth Low Energy (Bluetooth LE or BLE) version with got 8x broadcasting capacity and connectionless services. Thats' very nice. But it doesn't increase power consumption. Well well, that's something extremely nice. It's easy to add range or capacity, but usually that means that systems will require more power. I've already got a few EddyStone Bluetooth beacons, which are working very nicely broadcasting a few URLs.
• Also ordered a cheap Feitian ePass FIDO U2F Security Key and for comparison Yubico's YubiKey NEO. Just to have something to play with on my vacation. Even if I'm big fan of Strong password (shared secret) + HTOP / TOTP (as 2nd factor). I'm also wondering why some people don't get that Google Authenticator is not an independent technology. FIDO Fast IDentity Online - Universal Second Factor (U2F) - Let's see if installing pam-u2f will be fun. Terms like Passwordless UX (UAF) and Second Factor UX are used by some vendors. Device conveniently works as USB HID (Human interface device, aka keyboard).
• So much fail. Physical Web App for Android got updated. Now it doesn't show EddyStone Bluetooth Beacons which it did show earlier. Yet all the other similar apps do work flawlessly. - Business as usual, we improve the software so much it becomes totally unusable. - Found out the reason later, they now only accept HTTPS sites.
  

• About passwords and disabling paste: I think it would be highly beneficial in some cases to record password writing patterns & timings. If it's important system which requires high security. If key administrators try to login under "influence" whatever that is, drugs, booze, someone pointing a gun at them, extremely tired or very stressed, login would simply fail. Because you're not being "you" then. Requirement to write a bit longer phrase to check writing timings & patterns would pretty much ruin your day if your baseline is off. kw: password, passwords, security, cns, drugs, stress, influence
• Lossless compression with Brotli - A very nice article by Dropbox guys. Yet no news.
  
• Compression is also one of the fields where there's no "right solution" it always depends on so many factors. In some cases compression is very beneficial in some other cases it really ruins your day. One of the cases where we've been thinking about compression options is installing large system images over network. In some cases installation is done over 100 Mbit/s network with i7 CPU. And in some cases it's done over 1Gbit/s network with Atom CPU. This is more than enough to totally change what's required for the compression and where the bottleneck in the process is. In best case compression doesn't only save space, it also saves a lot of time.
  
• Checked some funny statistics. My SSD has been powered on for over 10000 hours in last three years. Yet it's still in full condition 0% worn. This will mean that I'm going to probably expire personally before this drive expires.
• Super long discussions with administrator friends about KVM, OpenVZ, LXC, and so on. Yawn, I could say. But it's not that simple, there's no single it's the best solution. It depends so much. I personally would go for LXC if and when running my own personal virtualized platforms. Also long discussions about if SSD / HDD manual allocation is better than SSD cached HDD storage system. Well well, it depends. Does the server proximity matter or not. Is higher bandwidth better than low latency or unlimited traffic. Is faster disk versus more RAM better. And many more other similar double edged sword discussion points.
• Fixed the issues with Physical Web. It seems that they now only accept HTTPS urls and even HTTPS -> HTTP redirection doesn't work.
• Cloudflare opened it's 83th data center in Moscow. Now it seems that some Cloudflare protected sites are being served from Moscow (DME) for Finland (HEL) instead of Stockholm (ARN). DME is lot slower than ARN as well as it raises some privacy questions is someone is worried about the Russian Privacy Legalization and Big Brother Laws.
  
• Checked Python 3.5.2 release notes. PEP 448 Liked unpacking generalizations, PEP 471 os.scandir(), nice. Yet another way to do it, but bit faster. Also the classic os.walk got a speed boost. PEP484 Type Hints is of course awesome. PEP 485, isclose is really nice. I've coded my own comparison routines at times just because == with floating points is almost guaranteed fail. There has to be some predefined tolerance to accept the value as being same. PEP 488 No more PYO files. Hmm. Sure. Doesn't affect daily operations, just looking for purity.
  
• Studied some Hyperloop related technologies: Kantrowitz Limit - Gas flow simulations in tube - Air bearing skis - Maglev using passive plate - Maglev using active coils - Required cooling neeeds. - Vacuum Energy Cost - Stations - Airlocks - Passanger Loading facilities - G forces - Thermal Expansion
  
• Something different: Hsiung Feng III - Explosively formed penetrator (EFP, SEFP, MEPF)
  

• Checked out LTE-M and compared it against Sigfox and LoRa. IoT future is here. 'Best' solution for depends from the needs of the actual application is being used for. In Finland Operators haven't forgotten M2M market at all. But maybe situation has been different in other countires. Depending from use case the main problem is the modem cost. But now you can get really cheap GRPS modems from China. Yet those naturally aren't LTE modems.
• Tesla Autopilot crash: A bad joke, but this somehow reminded me about: Darwin Awards movie, Autopilot Cruise Control scene. Afaik, Tesla Autopilot is an assistant, not a fully autonomous driving system. Which inherently means that it requires constant supervision. Using boats or planes autopilot won't either relieve you from monitoring where you're going to end up. Some of the news articles were titled with pure lies. 'Autonomous car crashed and killed', hmm, nope. Tesla isn't one. It's always important to acknowledge the true capabilities of a system and even then not blindly trust it. We all knew it was going to happen sooner or later.
  This is closely related to the many of the topics I've posted earlier where people trust systems without questioning those and well, we all know what's going to happen. Same rules applies to anything like fully automated stock investing. It can just go awry at times. Even if it would work perfectly in "normal market conditions" there are situations which will throw it totally off and that's the time when you're going to pay. Luckily that causes only loss of money, not loss of life.
  
• My First 10 Minutes On a Server - Primer for Securing Ubuntu - A nice very basics article. Yet it didn't contain anything new. Btw. With dynamic IP you can add your service providers IP range(s). It isn't perfect, but it's still much smaller portion of IP addresses than leaving SSH open globally. This is where the IPv6 helps a lot, because you've probably got just a few /32 ranges to add. If you want to know more details you should check out some of the proper OS hardening guides.
  
• Great post about TCP stack by Julia Evans - Sure. As with everything else, there's no perfect fits everyone solution. Many embedded devices use very small fixed Window with TCP. In general that was just same question as it is with standards, frameworks or something else. Use light simple one, use one mega bloated 'fits for everything' one. Or build your own. Which one happens to be the best solution. Also the build your own can be very highly optimized for your needs, but it also might require a lot more work than you initially thought it would. This is just like the question why we use TCP instead of UDP. most often it's just not worthwhile to re-invent something which is already working well enough.
  As example UDP networking and DHT turned out to be really hard for OpenBazaar team. You don't even know what kind of trouble you're going to get into before you try. There are just so many cases and 'problems' to deal with. Naive implementation might barely work, and it's guaranteed to be worse than Linux Kernel TCP.
  
• Breaking Android Full Disk Encryption - Let's see. Very long post, but it's all good and worth of reading. Yet again in this case again, it wasn't the magic AES-256 which was broken. But steps related to key management. It's with of noting that even using TrustZone and Hardware-backed Keystore (KeyMaster / TEE) protection didn't protect the key. I'm sure there are many other defected encryption products out there, it's just the case that probably no-one has bothered to take a good look at those.
  
• It's so nice to be IT department and tech guy and local support for all the friends & family. Helped people to change and use (proper) passwords, tether laptop and tablet with mobile phone. As well as consulted on VPN services & Torrent Seedboxes & Cloud Torrent Clients, install WiFi Repeaters so that whole terrace gets properly covered. - Something different than programming, operating systems and cloud servers this time.
  

• StartSSL.com / StartEncrypt Automatic SSL Certificate verification process was kind of broken (?). But funny or not, it's just about the definition. What's secure and what isn't. Why some sites do allow users to upload binary files and why some sites do allow users to redirect. Those are also 'vunerabilities' on the domain / site being authenticated. So in this case, it's so simple to say that it's broken. What's broken? It's not broken. The sites being verified are in one sense broken. It doesn't matter those are major actors. They'll let 'others to use their domain' which is also inherent security risk. As in my previous examples, I've said this is totally normal and this is going to happen over and over again. Is there some kind of official specification what's being considered 'secure' and what isn't? It's just the classic falling between cracks. Single thing isn't a problem, but when you combine those, it'll becomes a huge problem. Then there was the fact that the StartEncrypt client didn't verify the StarEncrypt server certificate. That happens, nothing new. There are plenty of programs which do not do that. And as I've said earlier even some banks recommend no to check certificates, because renewing those is pain in the.... Then it didn't check the MIME type. It was claimed that this allows getting certificates for any site which allows to upload avatars / pictures. That's yet another disputable claim. Of course those sites won't allow uploading such binary files, if those aren't being recognized as images. Or maybe some do, which means that those sites are also broken, and things are again falling between multiple cracks. Which sites allows 'binary file uploads' as images, which aren't images? Probably many. It's just simply too hard to check what the binary file actually contains, right? Incorrect file access rights (666) for the SSL private key is yet another example. Yes, file is readable or can be modified by anyone. But why would anyone give access to the server for non-trusted users? I know, it's yet another issue in this long line of issues. This wasn't anything new, it's always the same problem making hard things easy or easy things hard. There will be always problems at some level. It's also funny to notice that if security tools are this badly broken, what you would expect from most of so called 'normal' software? Hah, security is almost equal to none. Yet of course this shouldn't surprise anyone.
• Often I wonder what kind of people are designing the UX. Like in case where you first compose message and then send it and it ends up with some kind of error. Why you can't modify the message, there's only resend option which doesn't allow you to edit the message. I've seen this same problem with multiple apps. I just could say bleep about this. All you can do you can delete the message recompose message and possibly copy paste everything from the previous message to the new message and fix the error. But that's just stupid. Is it really so hard to allow editing message once it's placed in outbound queue? - Yes seems to be the answer. Technically that's nearly impossible engineers say.
• Quote from F-Secure: "Key endpoint protection strategies: * Hardening * Prevent malicious applications * Isolate applications from resources"
• Ring Learning with Errors (RLWE) - New cryptographic algorithms to protect against quantum computer cryptanalysis.
• Of course I had to check Quantum key distribution (QKD) too. We're living interesting times. I were actually familiar with this stuff, but it's good to remined you ever now and then about stuff you're not actually using.
  
• Google's Security article Experimenting with Post-Quantum Cryptography.
• Nice old article about Ring Buffers and implementations. - Nothing new, been there done that. But it explains well a few possible fail points if you're implementing ring buffer from scratch.
• More Tor attacks. Honey Onions
  

• Windows Media Player
  
• Desktop themes
  
• Video for Windows (AVI support)
  
• Windows SideShow
  
• Windows Defender
  
• Disk Cleanup
  
• Sync Center
  
• Sound Recorder
  
• Character Map
  
• Snipping Tool

Dumping stuff from backlog. ;) As fast as possible... (1 of 2)

• One invalid path caused system to hang without any error messages, even in logs. Logging doesn't reveal when task was finished, even if it was started. Lack of proper exception logging and handling. Caused huge mess hard to debug, even if reason was slight misconfiguration. As we know, making "program that mostly works require 1 unit of time", then making program which will work and handle all kind of exceptions correctly and also give clear logs / error messages can take 10-100 times the work amount. But some cloud environments require you to write programs so that the process / server can be killed at ANY TIME without any warning, it should not cause any problems what so ever. (Amazon, Google, etc…) If program works correctly, uses journaling and transactions, this should be the end result. If program is written badly without those primitives, result will be a program which works well in test environment ,but will end up causing total disaster in production when loads go up.
  How is scheduler state (done state / running state) defined? Does execution "end" when process tells it's done? What if process aborts / crashes? This actually is one of the parts what I have been especially working with my ERP projects. Because I have error handling for most basic errors which I'm expecting. But as we know, there are well, about 1000 times more errors which are possible, but I'm just not simply expecting those. I'm usually expecting errors like slight misconfiguration or invalid server name, login/password, invalid paths, file locking, network errors, data structure exceptions etc. But if there are other issues outside that circle, I'm catching ALL other exceptions and logging those. If the logged error message would have been clear enough, we would have immediately caught the problem.e
  In some cases I have gone to extreme lengths with checking all those listed errors, because I have wanted to make so robust solution that even if the process is killed at ANY TIME, next run will get things right. It checks for all temp files, doesn't ever write to "final file" directly, always uses temp files & lock files and prevents parallel execution by checking lock file time stamps. Naturally even this can fail if clock is played with or the locking process becomes really exceptionally slow and then is being executed again in parallel. It's hard to make programs which work correctly in all situations.
  
• NoSQL, Lock free algorithms (pitfalls), random bit flips
• Project budgeting, coordination and cost estimation, customers point of view, user experience, usability. Technical sales support tasks. Key customer responsibility. Customer contacts. Service desk experience. Wide ranged experience scope. Lets get this done attitude. Don't spend days thinking why we can't do it, if there's no real reason. Self guided, I'll study what ever I think I need to know to get the job done. Risk assesment.
  
• Wondered stuff about WiFi DropBox like WiDrop designs. I guess I've blogged this already. New Yorkers StrongBox.
  
• I should have written longer analysis about different anonymous P2P applications and what are the pros and cons of each design. Sorry, no time for that. (Freenet, GNUnet, Perefct Dark, Rodi, RShare, Share, Sumi, Winny, Nodezilla, Mute, Marabunta, OFF System, GitaTribe, Alliance, Direct Connect, Hybrid Share, OnShare, anoNet, OneSwarm, Retroshare, Galet, Turtle F2F, Waste, Kerjodando, I2P, JAP, Psiphon)
• It needs to be carefully monitored when Big Data analytics is beneficial and when it isn't. Should be quite obvious to any business. Article.
• Comodo provides P/ CSR S/MIME certs for free.
• Played with SciKit-Learn
• How Facebook shares your data with data brokers
• Operational Transformation
• No no time to play with Derby.js and Meteor even if those are interesting.
  
• Tor & HTTPS by EFF
• SIGNINT
• Reminded my self about off the record messaging, pfs, deniable auth, deniable crypto, malleable encryption.
  
• SSD device block generation, Could reveal enctypted hidden container(s). Because places in files are written repeatedly which aren't expected to be written often. Copy on write file systems might have similar problems.
  
• Gitian - Compare that delivered binaries correspond to source code.
• smtp_tls_policy_maps = hash:/etc/postfix/tls_policy
  Content:
  ferraro.net     encrypt
  gmail.com       encrypt
  mailsuomi.com   encrypt
• State of outlook & email serurity is here: status=deferred (TLS is required, but was not offered by host mx1.hotmail.com[65.54.188.94])
• Gmail TLS cert fails.
  
• When certificate fingerprint monitoring is on, delivery fails if fingerprint doesn't match: status=deferred (Server certificate not verified) - So email can't get delivered to wrong destination or without enryption even if there's MitM attack. Successful negotiation: xxx postfix/smtp[805]: Verified TLS connection established to s.sami-lehtinen.net[]:25: TLSv1.1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits) - fingerprint & Public key pinning.
  
• We should all have something to hide.
• Ad networks are delivering malware, nothing new. But still very annoying problem. Encountered malware at GrooveShark and SendSpace.
• Nonblocking Algorithms and Scalable Multicore Programming - Exploring some alternatives to lock-based synchronization
• Piledriver microarchitecture, Integer Factorization Optimization, MariaDB
• Reminded myself about: Mixmaster anonymous remailer, Anonymous remailer, Nym server, Van Eck Phreaking
• ITaaS, Software Defined Data Center
• Check your browsers Cipher Suite Details.
• OWASP TOP 10 list, legendary. 
• Misconfigurations do affect affect all security solutions, not only computer security. One building just had it's lock system mis-configured. Anyone having key to the building, could access all areas. I really mean all areas, like telecom and power distribution rooms and any spaces occupied by other businesses. Nothing new, just a minor configuration issue. It's ridiculous to first spent a lot of money for high end security system(s), and then totally mis-configure those. 
• VSRE standard - Yes!
• Unhosted idealogy, I kind of like it. But I doubt normal users will like it too much.
• Played with compressing data on paper. Practically useless, but nice play. See data compression. 
• EAX mode is better than CCM yet GCM is really complex. AEAD, AE.
• TLS/SSL RC4 beast ephemeral keys, ecdh edh, session resumption.
• EU data protection requirement
  
• Coroutines and concurrency
• Ladder of Abstraction - Yes, check it out. It's worth of it.
  
• HubSpot SaaS 101 lessions
• Bootstrapping a software product
• SifterApp, simpler bug & issue tracking
• STARTTLS - No comments, just dumping
• How much does the selected algorithm change the end result? Maybe a lot. Jump point search. It just makes a huge difference!
• At least I have learned a long time ago, to avoid doing something complex cool and smart. Doing the simplest working solution is usually also the most reliable solution. Any smart tricks which add any complexity to logic of program are going to cause a lot of trouble later. I have very bad personal experiences even from most simplest solutions. Like comparing a few fields to few other fields, with logic. It was too complex for programmers to implement correctly and lead to annoying bugs that persisted for months. All this could have been very simply avoided by dropping the comparison logic and using simple unique ID on every record. It's worth of noting that most probably anyone later modifying the code would break that logic again, even if it would work.
• zRAM / Compcache - Compressed memory, nothing new afaik.
  
• Chromium Disk Cache structure - Small data is but in blocks - Yet I remember seeing some other article (later?) where they implemented SIMPLE DISK CACHE which did always use files for simplicity. It's funny how optimizations can be bad at times. Just like should MORK file format (msf) be used as index file for standard mailbox data... Or is simple maildir approach better? I'm using both. Or maybe data just should be simply in SQLite database? Who knows. Does it really matter if it works out?
  
• It's nothing new that Telcos require all confidential information to be encrypted... I wonder why, I'm sure their security stuff must have known about this. Also, encrypting everything which isn't classified as public just makes sense anyway.
• Recursive PostgreSQL queries
• Studied documentation why Skype abandoned original P2P model and started to use cloud servers instead. Everything was done in very reasonable manner from product managers and normal end users point of views. - Sorry, no link.
• Securing the Cloud, Cloud Computer Security Techniques and Tactics - Vic J.R. Winkler - Cloud Security, I really liked Epic Fail sections which showed what kind of epic failures there has been with each kind of security issue type. Made me smile. - type 1, type 2 vm. VirtualBox, Parallels, Virtual PC, VMware Fusion, VMware Server, Xen, XenServer. LXC (Linux Containers), BSDjails, OpenVZ, Linux-VServer, Parallels Virtuozzo
• Gallery how different machine learning algorithms detect patterns. Excellent stuff!
  
• FileDropper advertices 5 GB files, but actually 1.7 GB file is too big. - Btw. They'll still have similar issues in 2015. Upload large files and those somehow disappear or something similar.
• Cross & double monitoring of services - Always use external and independent system to monitor other systems. Also deliver alerts "out of band" if possible. I've seen it happening so many times that when systems go down, there's no alert... Because the monitoring & alerting system went down too.
• Elliptic Curve by NSA
• That's why they take memory snapshot first, which is trivial with VPS and then pick encryption keys from it to access encrypted volumes. This is well known method and works with pure hardware machines too with physical access. It's great question when you get to server, to shut it down or leave on. If on, it could destroy data, if turned off encryption keys are gone. I think it would require some individual case analysis before deciding which one is better approach.
• Nightweb protocol
• Actually it seems that I blogged this later. But back in 1.7.2013 I also made memo which clearly showed that when you browse using DuckDuckGo with Dolphin Browser data is flowing over non-secure (non https) channels. I do have packet logs. But because sanitizing those would take too much time, I'm not going to post those.
• Excellent chart from XKCD guys. Is it worth of doing? - I actually have printed this one on my wall.
  
• Motorola is listennng in. Android phones do spy users.
• Never trust Facebook. - Uhhm? Who would be foolish enough to trust it?
• Functionality and differences between BeiDou, Glonass, Compass, IRNSS, GPS, Galio satellite navigation systems, signaling and design. As well checked out legacy eLoran and TIMU aia simple Inertial Navigation.
  
• When it comes to the price tag for BI, a lots of people are shocked to learn how much some companies charge. For large companies 50k is kind of a low end instillation, and BI can cost over half a mil. However, the question shouldn't be how much is BI going to cost me, but how much can I expect to save/increase profits thorough this tool?
• Android Master Key - Android security is seriously broken.
• Studied documented Priority Inversion software problem with NASAs Mars Pathfinder spacecraft software. - Nothing new.
• USB Hell - Drivers, sockets (4 different), charging & power issues etc, different cable connector types etc.
• One thing has become clear to me over time, especially in the current financial crisis,” she said. “No matter the job, most of us no longer have job security. Our labor is replaceable.” - Yet another reason to keep studying.
• PGP Tutorial - Yes, you should already know everything mentioned in this document. If you don't, just go and read it.
• Prism Break
• Additional VPN transport - TCP over WebSockets over HTTPS. This is great way to access all services when guest networks have TCP port restrictions. Yet much faster and more efficient (less protocol overhead) than tunneling over DNS which I did setup earlier
• SIM card insecurities
• Liked Hong Kong, Singapore, Sydney, etc. Peking airport WiFi was strongly authenticated (passport), but poorly secured. Which means that I could have hijacked anyones identity and made them look guilty. Afaik, this is very bad way of doing things.
  
• Death of public key encryption? - Sounds even more viable in 2015.
  
• Applied practical cryptography
  
• The AEAD modes combine stream cipher modes with MAC constructions that developers don’t have to think about.
  
• In computer security, a covert channel is a hidden signaling mechanism. Attackers exploit covert channels to leak messages across security boundaries . Side channels are the flip side of covert channels; they’re actual signaling performed unexpectedly.
• Bank allows reusing authentication codes in case of some errors...
• Tox.im - The way of secure (?) future messaging?
• Dynamic real-time pricing
• Probabilistic programming and Bayesian methods for hackers
• HTTP/2.0 initial draft
• NSA monitors everything  
• GSMA:n Wallet-POS APDU, GlobalPlatform Security Domain
• SSL/TLS broken - Gone in 30 seconds
• Checked out Mifare Ultralight and Ultralight C variant with crypto
• Enjoyed psftp key management using registry editor... So wonderful.
• SQL injections - Joining Union query-based injection, squeal, error-based injection, boolean-based injection, time-based blind injection. 
• True Hacking, hacking hard drives! - Excellent stuff!
  
• Private / Public surveillance relationship - Just what I have said. If something can't be officially done. You'll just find a way to get it done unofficially.
  
• Hidden Services, Current Events, and Freedom Hosting - Multiple hidden Tor services taken down and some of those tried to plant malware on visitor systems. Afaik, some of those pages planted back then do still exist (2015) with that yellowish background and some kind of frame near the top.
  
• Watched BBC what's killing bees
• Unprofitable SaaS business model trap
• Studied a nice list Linux TCP related parameters
• Why web apps are slow (?)
• Studied & Configured firewall: ICSA certified ipsedc & firewall, USG (Unified Security Gateway), SPI, SNAT, DNAT, BWM, ADP, IDP, AV, AP, CF, AS, Routing, HA. In long format, destination nat, routing, stateful firewall, anomaly detection and preventation, application classifier, intrusion detection and prevention, anti-virus, application patrol, content filter, authentication, system management, logging & monitoring, anti-spam, threat database, high availability, vlan, ipsec, sll, l2tp, real-time, dynamic malware protection, IPV6, security zones, dual stack.
• Computer insecurity
• Enabled PFS for my own server https://s.sami-lehtinen.net/
• Lavabit, Silent Circle, Lavaboom, Pidgin, Empathy
• SSD Endurance myths
  
• End to end (E2EE), pre internet encryption (pie)
• Python Statistics - PEP-0450
• Watched GNUnet video - Nothing new in this video, I have been liking GNUnet project for several years.  But if you don't know GNUnet project, check it out.
• The GNU Alternative Domain System (GADS) (GNUnet)
• Of course checked out HyperLoop!
  
• ENISA
• I really like teams which get stuff done. Quick iterations, meetings, let's get it really done. Push and do. Test, iterate, quickly! I hate projects where every single question takes days and test iterations and stuff takes weeks or months. I love the feeling when stuff really gets done, and everyone got the same mood. I can do it. - Not the attitude, that I don't know if I really need to get it done, if someone else could do it some day or ... Aww!
• Seems that 42registry is working again. My postfix also handles mails to myaddress @ s-l.42 or myaddress @ samilehtinen.geek properly now.
• I've got separate email address for GPG only messages. If any email which isn't encrypted to my public key is sent to the address, it bounces back with link to my public key.
• Windows server access policy management.
•  Yes, not using cache at all would make everything very slow. I'm now of course talking about using in session memory cache. If it's too small you can reconfigure it using browser.cache.memory.capacity parameter with Firefox. With fiber I never use caching. But yes, with 512kbit/s connection I unfortunately had to use disk caching too, to avoid re-downloading anything I simply could. But of course in that kind of situation and configurate you're really aware that you're not destroying all data between sessions. For privacy virtual machine with hardened configuration + tor is good idea. Otherwise there's no reasonable expectation of privacy anyway, as they're saying. In technical terms, there are so many ways to track users who do not harden those, that there's no reason to expect any privacy. As we have seen with all these NSA discussions, all technical options were pre-known already. You don't know if sites use some techniques or not, but it's reasonable to expect that they do use at least all publicly known techniques. And possibly some unknown. So making attack (or tracking) surface as small as possible, when looking for privacy is reasonable. Maintaining any data between sessions is just stupid. Always boot clean virtual machine, which is similar to other virtual machines, is best approach. Otherwise there are tons of things they can do to track you.
  Btw. even if browser keeps cache, you can always clear storage paths.
  One of things that doesn't seem to be known to many users is that many databases contain 'deleted' data for long time. They just don't think about it. Just go thorough all files stored by browser, you'll end up finding stuff that you woulnd't expect to be there, if you're naive. Right attitude is to expect everything to be stored always, and take proper care to destroy data when it's required. This is just like the issue with SSD drives. If you write something on drive, you wan't to destroy. There's no sure way to destroy the data from drive, without totally physically destroying the drive. You simply don't know, if the controller has written data to cell XYZ, and then re-mapped XYZ to somewhere else. Just overwrite it approach does not work in this case. And you can't even guarantee that the manufacturer tool could properly erase that cell.
  Just final words. Etag doesn't have anything to do with "images", it's not tied to content-type at all. Next week I could release "css" tracking exploit, which uses etags, which is kind of css checksum. Uh...
• RFID readers, NFC tags
• PostgreSQL basics by example
• IronPigeon - Yet another secure communication protocol
• SEO and Google+ correllations
• Low level flaws in 3G Modems might allow hijacking computers. SMS shell fuzzing usb internet modems.
  
• Circumventing Network Backouts @ Schneier
• Especially traps were important when processing money. Of course you can work around those using alternate code, but why bother, when there's existing library. Python Decimal Yes. I've known this for 20+ years, but this is just reminder. Yet another coder used floats for money and as we know, it's sure way to fail.
  
• People often laugh at bad password policies, but sometimes password policies are just catastrophically bad. Like in some cases, shared passwords are used so widely, that it becomes impossible to renew the password, because nobody knows how to inform everybody who needs the password. That's one of the ultimate fails I have seen happening. No no, we can't change the password, because we don't have any idea how to inform the people who need it. Afaik, password should have been changed ages ago, and should be changed monthly or so. Of course it would be better not to use shared password at all, but in some cases that's also too difficult goal to reach.
• Otr main key collision would mean that random numbers are bad, really bad.
• IT security stuff: SSH shell accounts, when SFTP should be only available. When informed about this matter, they don't care, it works. Rights elevation exploits. Passwords stored at end users workstations. Not in secure environment. Exe files which are run with system account, but all system users do have full write access to those files. etc... Business as usual.
• Lean software development, Lean Integration
• Multi-process listening sockets
• Reddit lessons learned scaling to 1 billion users
• SQlite3 - Partial Index, Next Generation Query Planner
• Checked out Zoined Retail Analytics in detail (with demo account)
• Bruce Schneier's blog: " We're always making trade-offs between security and efficiency. "
• iDRAC IDRAC DELL Server stuff, Remote management, etc, VNLP.
• IPsec is crappiest protocol ever, at least in real world.... Yes, I'm having serious problems with it AGAIN. No words for this "#¤"#4.
  
• F-Secure securemail requires that password contains symbol, but " isn't valid symbol, how lame!
• Dissent accountable anonymous group communication
• I received email from F-Secure: "
  This is a secure message, click that link right now!
  That's just one more reason to use individual email address for every service & communication instance. You'll immediately notice that the message is out of context of that email address it was sent to. I've been also receiving UPS can't deliver packet notifications and other scams, which are only asking you to click link provided.
  In this case this message actually is legit. Anyhow the first impression is quite much, yet another scam, delete.
  I just had to spent quite a while analyzing message headers, body content and link destination  & source servers,  AS numbers, IP-addresses, before I decided to open this link with my broser."
  
• Watched many PyCon CA 2013 videos
• Great example how accurately people can be tracked using mobile phone data, even without GPS, see the cell sectors marked on map.
• Finished reading book(s) Securing The Cloud, Innovators Dilemma, Enisa Cloud Security, PCI DSS Cloud, HIPAA check list, PCI SSC Quick Reference, The Dangers of Surveillance, "
  Bitmessage: A Peer‐to‐Peer Message Authentication and Delivery System Jonathan Warren", Go Faq, 57 startup lessons.
• Replaced my encryption key with 4096 bit RSA key, signing key is still old 1024 bit DSA. Still waiting gor Elliptic-Curve keys to pop into main release og GnuPG.
• Studied Linux Kernel 3.11, especially Zswap seems to be very good trade off between slow I/O and fast CPU.
• Studied distributed file system lustre.
• Found a few sites which looked like a scam. I had to use wget to examine headers and content before opening the content with traditional browser.
• Nice post about using LXC for network isolation.
• Played with DebianDB in VirtualBox environment. See dbtechnet.org for more information. Databases used MySQL DB2 Express-C, Oracle XE PostgreSQL Pyrrho.
• A few minor fixes in one project using Tomcat / Hibernate / MySQL made whole project 16x faster.
• One coder used framework he didn't completely understand. Originally there wasn't password requirement for users. But when site got more confidential data password feature had to be added. Well how it was done? When user gave login and clicked ok, came page asking for password. But if you changed url at this point, everything worked. When I checked the actual code, the user form logged user in and password form logged user out if password was incorrect. Oh boy, what a joy.
• Keccak (SHA-3) slides at NIST
• Toyed with different index types and partial indexes and stuff like that with SQLite3 and PostgreSQL just for fun. Because PostgreSQL 9.3 was released.
  
• Blog more about ... ERP stuff like: inventory reporting, financial reporting, sales commissions reporting, inventory management, purchase orders, sales statistics, customer accounts, distribution logistics, EDI, credit checks, electronic invoicing, invoice processing, EDI, stock management, dynamic (time-based automatic) pricing, web shops, system architecture, enterprise resource planning, SaaS, delivery reliability analysis, supply chain management, SCOR-model (Supply-chain operations reference), performance measurements.
• Great source for studies edX
• SSEBITDA for SaaS business profitability measurement
• Beyond passwords, new biometric identification methods.
• A very nice checklist for authentication & passwords.
• Some customers just got "Total illusion" about project scope, schedule, costs, etc... But I guess this isn't news to anyone.
• I'm using SQLite3 for my many of my small utility projects to track processed messages, just as Bitmessage does... Why? Well because usully the main database doesn't have any flags, tables or reserved fields in database I could use. So I use another database to track and compare changes in another database. Isn't that great? Why I won't add fields or tables? Well, it would break several compatibility things, that's why.
• OpenVPN uses the OpenSSL - Security Now show said that OpenVPN is not OpenSSL, it's more secure. I guess they don't know too much about OpenVPN at all? Because it's directly based on OpenSSL. Ouch.
• I don't know what Hushmail's web font does. But it doesn't idle properly. If you leave hushmail tab open in mobile browser (Android), it'll drain 50% of battery in 45 minutes.
• It's better to verify certificates fingerprint than it's signed by "official trusted authority". Because fake certs could easily have trusted signature, but it's much less likely that they got right fingerprint.
• Studied Ulam Spiral for network node partitioning purposes.
• Checked list of goals for Phantom project and read it's white paper. Goals of the project do sound really great. [ Complete Decentralization, maximum Resistance Against All Kinds of DoS Attacks, Secure Anonymization, Secure End-to-End encryption, Isolation from the Normal Internet, Protection against Protocol Identification / Profiling, High Traffic Volume and Thoroughput Capacity ]. Documentation contains important aspects like: routing paths, addressing, network database, encryption and authentication, and technical description. As well as the most important part, which makes Phantom better(?) than I2P, TOR or other existing anonymized file sharing software.
  
• Hosting backdoors in hardware by Oracle.
• SQLite4 LSM Design - A very interesting read.
• Lightly checked out Incapsula - Yet another CDN.
• Checked out zBase - Yet another large distributed key value storage.
• Reminded my self about Kademlia, Chord, Tapestry and Pastry internals.
• Studied: Gossip Protocol
• BitMessage Streams / Partitioning, how to prevent all data spreading to whole network. Yet maintaining the benefit of broadcasting not revealing who's the actual recipient of the message.
• Verified TLS connection established to s.sami-lehtinen.net [x.x.x.x]:25: TLSv1.1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits) - Seems to be working.
  Received: from x.x (x.x) (using TLSv1.1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "x.x", Issuer "x" (verified)) by s.sami-lehtinen.net (Postfix) with ESMTPS id X for  <my-mail@sami-lehtinen.net>; Thu, 19 Sep 2013 21:58:07 +0300 (EEST).
  Anyway without cert check / fingerprint, using starttls (smtps) still foils passive attacks.
  
• Donated money to EFF, PBS, SomaFM. Actually it was quite funny process. Because PBS didn't allow donations outside US. I had to send them feedback that I'm having a problem. I can't donate money to you. Nor I can watch your shows legally, I have to pirate those. I'll download their shows and donate money, that should be ok, everyone should be happy.. They fixed it, now they allow global donations using credit card.
• Downloaded Dell iDRAC6 software and had to run it as standalone app using webstart because it simply didn't work properly with browser. Anyway, I'm glad I got java background so it was quite trivial to start the app using the jawaws.jnlp file from command line. Main trick is that the credentials needed for access are inside that laucher packet, so it has to be fresh.
  
• Google knows WiFi passwords. - Isn't "central command cloud just so beautiful thing?"
• Interestingly if x is None is faster than if x == None when using Pyhton 3.x
• Accidental downtime caused. Ouch! One image hosting site had interesting stuff, using wget was way too slow, so I wrote my own version. I had it configured to work in two steps, first I go thorough all index pages and collect image urls. And then next phase is to fetch the actual images. Simple stuff. Because I've been playing with 8 thread CPU lately (i7) I had to try what I can do. So I wrote small Python app which utilizes both, multiprocessing and threading. Only problem was that I was running 16 processes and 32 threads per process. No, it wasn't problem for my system, but it was clearly a major problem for that small image hosting site. 512 concurrent HTTP connections fetching large images over keep-alive connections shouldn't be a problem, but in this case it was clear that it brought their site down. I wonder why they don't have rate limits or limit connections / IP address etc. But that was the unfortunate result of that. Of course I stopped my fetcher and tuned it down a bit. But it was funny to notice how easily some sites simply stop working, even if you're not even trying to distribute requests causing the overload.
• Thoroughly studied 1024 cores - concurrency, parallelism, wait-freedom, lock-freedom, obstruction-freedom, algorithms, waitfree, lockfree, algorithm.
• Had a Google I/O 2012 and 2013 marathon. It's much more informative than watching random TV soap opera or other "junk"
• Read ARM64 and you.
• Studied GCM mode. AESNI
• I just hate some Android bloat as much as some Laptop vendors which pre-install all kind of crap. No thank you. I want as bare system as possible, and then select just a few applications which I actually do use.
• Had long discussions about developers if UI buttons should have single role or if those can be multi-role buttons depending from the user situation. Some key buttons should be static, but rest of it can well be dynamic. Just like the few key buttons on Android (back, home, properties) and then the touch screen is "dynamic multi-role button section".
• Lot of discussion and articles about fingerprint security and so on. But as said this brings it all together, fingerprints alone shouldn't be used for authentication. It's only identity.
  
• Win-Win Consulting - How to deal stuff so everyone will be happy. Been there done that.
• Article how to switch to HTTPS for free. But as said StartSSL certs aren't free for businesses.
  
• Use CheckTLS to confrim your SMTPS secure email settings, yes you can use it to check also your service provider even if you aren't administering the systems.
• People who are able to manage private and public cloud as well as communication between departments and understand business needs, integration and system architecture, will be in high demand.
• Privacy and Trust - Bruce Schneier
• Facebook email had it's problems: conversation with msgin.t.facebook.com[66.220.155.11] timed out while sending end of data -- message may be sent more than once.
• Detector.io project - Detecting possible MITM attacks with Tor
• It's true, after I posted my latest blog post (to blog) and posted it publicly to G+. It was then indexed by Google in less than 15 minutes, even if I haven't updated my blog for months. It's clear that Google+ posts trigger Google crawler and indexing activity. Posted a bunch of other stuff to see if it works and yes, it seems to be working.
  
• NSA generates map of populations social connections? - Ugh, no news. Sociogram analysis isn't anything now.
• Amazon RedShift What do you need to know? 
• Telehash yet another P2P secure MESH protocol.
• Freelan yet another VPN software.
• tinc yet another VPN software. Compression & ECC.
• Robots2 - Extended standard for robot exclusion - Updated my own robots.txt's accodring it.
• RoundCube WebMail all contains 2700 files... OMG! No wonder there's some "random seek". That's one of the reasons why I had to get that SSD, because starting simple app like VLC took for ever, due to sheer number of additional libraries in files to load.
  
• How NSA attacks Tor.
• Installed latest RoundCube and converted it to use SQLite3 instead of needlessly heavy MySQL. - Everything works well.
• Had deep technical and analytical discussions about SQRL with a few friends. 
• The problem with SQRL article which our discussions were based on.
• Lol, found yet another case where Windows IIS FTP service were continuously used to access server over Internet world wide using Administrator account. RDP was also available. Hmm, so much security fail here it's hard to describe. 
• Had to tune for a while to get Windows 2008R2 virtual domain support to work with IIS FTP.
• Anonymity is hard, great reminder. You're probably deceiving your self if you think you're being anonymous. OPSEC
• The problem with timestamps - Nice post, time is hard, even if it's easy to think it's quite simple thing.
• /dev/sdb1 on /mnt/s1 type ext4 (rw,relatime,data=writeback,commit=300) - Great way to get better than SSD burst write performance on traditional disks. Just use writeback mode and long enough commit time. Btw. This also works great for slow USB sticks and stuff like that.
  
• Checked out NFTables which is going to replace (?) good old IPtables at some point in future. 
• Checked out Quantum Algirthm - Made my head hurt. No, I'm not mathematician, too deep and theoretical for me.
• Great TED talk, why privacy matters.
• About buggy software. I just remembered classic Laser Squad targeting fail. In the game, when you were shooting far away, it computed the hit accuracy based on distance, weapon and shooter. But main trick was that if there was straight linear line from you to the target. You didn't aim 100 blocks away. You aimed to the first available block to that direction. When the shot was fired, it continued to travel thorough that point forward, because it didn't hit anything. This made it virtually impossible to miss. Let's make ASCII demonstration.
  YT------------------------------------------------------------------------E
  So instead of You shooting at Enemy you shot aimed and shot for Target point, and the projectile then just traveled to the Enenemy and hit it for sure. Nice fail, right?
• Added custom HTTPS rule for my own server. In this case if I forgot to write the HTTPS and HSTS isn't primed my client will still use HTTPS.
  
   
    http://s\.sami-lehtinen\.net/" to="https://s.sami-lehtinen.net/"/>
  
  Using the custom rules allows you to guarantee HTTPS access for any other domain you wish to.
• Played with Python BitString, just for fun. I haven't needed it this far, but if I need it. It'll be nice tool. It's good for situations where there's long list of something, which is
  is in  contiguous space but there are different boolean states for entires. I just often use dictionary which contains integer ID with Python. But I do know it's not space efficient representation when data
  set gets larger. Good part of this that it's able to handle gaps without problems in case IDs aren't in contiguous space. If space is contiguous list with booleans would work too, but it uses 8 times the memory for same representation.
• This is old stuff of course, but I reminded my about it. It's still a very nice optimization. Perfect example how things can be made faster, if you just bother to think for a while.
• I just noticed that LG servers accept absolutely huge HTTP post data. Several 25 gigabyte files were posted successfully in parallel. So who says that you would only send them the filename or tv channel
  information. You can stream them full hd tv-shows and send full bluray rips to them in real time. curl -F "file=@your-show.extension" http://their-http-post-url/ I just sent them a lot of data and they accepted it happily and didn't even ban my IP. Which is nice, no reason even to use multiple IP's for sending. Maybe I'll send more? Or what about if million users would send them a few terabytes or even bit less like hundred of gigabytes. From now on I'm sending them everything I'm watching. So if you got seedbox with free (or near free) bandwidth, just send a few copies to them of everything you're seeding.
  I personally currently paying for about 30TB of traffic per month, and I'm only using something like hundreds of gigabytes. So now I got great destination for my excess bandwidth.   
• This reminds me from time in mid 90's, when I asked for OS/2 from my friend, and he uuencoded whole cd image and sent it to my email box from university servers. My ISP wasn't quite ready to receive about 1 gigabyte of email, which was absolutely huge amount of data back then, they got quite mad. Btw. It was the time when even ISP were using 10 megabit coaxial ethernet networks.
• Yet again I kept Internet engineers hand, and told how to configure routing table and interfaces, because "things just won't work". Of course those won't work, because you have invalid configuration. It's not so hard after all. Or it is hard, if you don't have a clue what you're doing, even if you're supposed to. 
• Decompiled some .class files and inserted my own parasite thread inside the program to do stuff I wanted to do. Worked great. I even added automatic scoring features, spam text messaging and so on. Because it was "flood cast" game, which sent messages to all other peers, it nearly crashed whole game because they didn't have proper rate limits and my flooding of X Mbit/s to server caused it to flood 1000's X the bandwidth out. Duh! I'm sure there were many clients which weren't able to even handle that data flow even if the server wouldn't gotten swamped. Then the queues would simply have eaten the servers resources, if those wouldn't have been properly limited. So the server itself created amplification and resource consumption attack against itself.
  Well one of my friends did even better. He reverse engineered whole applications communication protocol, including some protections they had there and wrote own client using C.
  It's good question which solution was better, at least mine was simpler to execute, as well as it allowed me to still use full client with just my own tweaks.
  
• Studied covert channels.
  
• Cloud services are natural way to go. Exactly same thing has happened in other sectors much earlier, and it's nothing new. drinking water production, electricity production, banking, etc. Self-producing something is only viable, if scale is large enough or needs are very specific and outside commonly provided services. For this reason some industrial plans which require a lot of water or electric power do have own systems for producing what's needed. But for most, it's good idea to outsource. It's just like running own data center in office closet. You can do it, but it's probably much worse option than out source it to cloud. Of course Facebook or Google could use cloud servers from Amazon, but scale is huge enough, so it doesn't make sense. At least in Finland many companies which are selling server hosting, server management etc, are currently also promoting cloud services from Google and Microsoft, even if it's direct competitor to their own services. Only difference is that Google and Microsoft probably give them economical incentive as well as, services are after this still cheaper for end user than services provided by local service provider.
• 2FA won't protect against cookie stealing.
• Microsoft povided Office 365 is ok'ish. But the sharepoint which came with it, is ridiculously slow.
• I love encryption, but I usually use it for backups or only highly private information. Because error in key or loss of keys etc, could render a backups / data totally inaccessible / unusable. But for that reason, I usually take from most very private data a backup copy to another off-site location, which is encrypted using different key & technology. So I'm hopeful that if something bad happens, at least one copy is restorable.
  I never trust cloud services with the only copy of the data. Either it's local data, which is backed up to cloud (encrypted) or it's cloud data which is backed back to office (encrypted). Both backups data and connections are of course bot encrypted. And all with different keys. If data is encrypted at the original source, is secondary question, it might be or might not be, depending what it is.
  Even my own email server is backed up off-site daily, with encryption, and proper backup history (for months).
• One message routing protocol was written so badly that it was only able to handle one remote connectivity point. The project was really complex and basically unmaintainable. When there was need for multiple destinations what did I do? Well, of course I wrote simple message store and forward solution with multiple outputs. Now the original project just forwards the data to my python forwarder, which then pushes it as ordered but asynchronous process to multiple destinations. Everyone is happy and things work well. Related terms, SOAP, WSDL, Python, WebService, routing, router, fork, splitter.
  
• Something different: Russian and Chinese air craft carriers and radar & weapons systems being used. Especially story of Liaoning is interesting. 
• Broadcom PhyR - DSL Physical Layer Retransmission, Bit Error rate (BER), Reed Solomon (RS), coding gain (CG), forward error correction (FEC), error correcting code (ECC), Impulsive Noise Protection (INP), Repetitive Impulse Noise (REIN), Interleaving, Error Propagation, IPTV.
  
• Chocolatey - APT-GET for Windows? Yes, I've always loved proper package manager.
  
• BoxStarter - Easy Windows environment installations.
• Played with SqliteAdmin and SqliteBrowser. I think the browser one was better and clearer for my humbleneeds. Admin one is better if you got more stuff to do.
• Authenticated SMTP might be blocked in some cases. Elisa (Finnish ISP) seems to be blocking even authenticated SMTP outside from their network. So operator is basically forcing customers to user their SMTP out relay.
  
• Lars Knudsen, Accumulated Block Chaining(ABC) mode, Carl Campbell, calledInfinite Garble Extension (IGE) mode
• Cryptography is hard. - So you want to crypto?
  
• Big data quality issues ... Just so common, daily stuff, without constant tracking, monitoring, auditting, trail logs, reporting, it's going to be just garbage.
• Block, hash, delta, data compression, de-duplication ... Duplicati, duplicity and generic rsync etc related discussions.
  
• Batch, realtime, near realtime, integrations, etc. I thought I would about this... but I guess this post is already long enough.
  
• Just want to remind people about the fact that something what is being triggered asynchronously after something has happened is NOT realtime. It seems that some people are pretty confused about this.
  
• Had interesting discussions with ThinStuff guys. Discussions were related to update automation, security, authentication, notifications, RSS feeds, timing and so on.
• Keywords: Customers point of view, user experience, usability. Technical sales support tasks. Key customer responsibility. Customer contacts. Service desk experience. Wide ranged experience scope. Lets get this done attitude. Don't spend days thinking why we can't do it, if there's no real reason. Self guided, I'll study what ever I think I need to know to get the job done.

Sorry if there's something unclear. I just dumped this stuff as fast as I could. Now my backlog contains anymore only 385 entries for 2014. ;) I'll be posting more soon.

• Did a set of training tasks & experiments with Kali and Tails. Just to maintain capability and skills if and when required.
• Very nice article Data structures for external memory - Liked it, timings, measurements, different approaches and solutions.
• Linux kernel synchronization primitives - sequential locks - That's one way of doing it. Using counter and checking it is very efficient. Yet it can lead to situation where lot of resources are wasted because tasks need to be repeated. Of course this is one of the problems that such locking could cause. This is one example of 'opportunistic locking' (OpLock) as it's called in Windows or Optimistic Concurrency Control I've written a lot about it earlier.
• A list of Bitcoin related computer security incidents - Btw. This is quite awesome list. 38 incidents listed so far. Race condition, account take over, social engineering, backups, application vulnerabilities , insiders. All kind of attack vectors were used. Often even one trick isn't enough, they combine multiple to get around the obstacles preventing a successful hack.
• Reminded myself about WiFi interference - troubleshooting basics. Nothing new. I knew it all. But if you are having trouble with WiFi, it's worth of checking this out.
• Real–world HTTP/2: 400gb of images per day - That's one of the reasons why I've implemented HTTP/2 (h2 and h2c) for my services.
• Asciinema - Why Python is better than Go - They listed all the stuff why I also really like Python.
• Mr.Robot Easter Egg for S02E01 - Nice, nothing surprising yet. All 'standard' and well known encodings.
• Microsoft: Our search warrant case: An important decision for people everywhere - This is interesting case. And something we've been waiting to see out. I personally think this is the only sane way to get it done and follows the Privacy Shield policies.
• Telegram E2EE encryption is much faster than WhatsApp's. I guess WhatsApp is doing some overkill public key encryption repeatedly on every message making it slow and consuming a lot of battery and CPU resources. Afaik, that's bit excess. Like generating new 2048 fresh bit RSA key for every individual message and signing it with long term RSA key. Aka, ephemeral keys for every message. Yes yes, I know. There's documentation available which I could read. But I'm not that interested right now. I'm just reporting poor UX which in this case isn't great because of the slowness. Telegrams Approach where key is renewed using 'sane' interval is much better.
• Something different? Checked Russian Tupolev Tu-214R ELINT aircraft specifications.
  

• Read more about ISO/IEC 14443 tags. I've been writing some NFC related integrations. But I haven't had to deal with the low level stuff ever. Usually the application just uses "unique blob" and I don't care what that is. I got to the page because I were interested about ATQA SAK and ATS values + wanted to know how long tie UID is. Even if some apps seem to call it Serial Number? Which basically is the same thing. I'm glad that the password protection features worked well with NFC tags. As I've reported earlier some EddyStone BlueTooth Beacons are totally broken and won't basically allow setting any other than the static default password. Which of course is a major security fail.
• Configured few tags to configure Guest WiFi + Open Company Web Page whenever touched. That's pretty neat. Printed a few standard plastic credit card size cards with NFC symbol and WiFi information in reception and meeting rooms.
• Added internal tinfoil lining to my wallet to prevent remote NFC card reading without taking cards out of the wallet. It worked really nicely.
• Watched long documentary about intelligence services and covert action and sabotage they're taking. Small groups of hackers, seeming to be independent actors. Naturally most of interesting questions and topics were classified and not discussed publicly. KW: Zero Day Attacks, Intelligence, Espionage, Sabotage. Quick Money, Hacktivist, Sending Political Message, Nation-State Actors, Cyber Weapons, Cyber Command, Air Gap Jumping, Weaponized Code, Advanced Capacity and Capability is highly Classified, International Law. Everything you can get away with is ok in Cyber Realm. Cyber-Attack Targeting and Intelligence. Critical Infrastructure Vulnerabilities. Botnets, Destructive Activities. Computer System Knock Out, State-sponsored Cyber Sleeper Cells, Data Exfiltration, Infiltrated Command And Control Systems, Nitro Zeus, Attribution hard.
  
• New article about CloudFlare: We Have a Problem. Well, I think the article didn't provide any new information. That's just how CloudFlare works. For some cases it's ok, and for others, it isn't. I do use CloudFlare for a few free sites, but none of business sites are using it. Also SPAs got mentioned. MitM risks, maliciously intercept traffic, dragnet interception, TLS/SSL breaking, and so on.
• The same issues apply to running your own email server. Sure it can be hacked, of course. But it still requires someone bothering to do so. Instead of collecting your data directly from 'cloud hosted email' as the usual mass surveillance.
• New broke out that health care system allowed people to watch highly confidential diagnosis information of individuals without that access being logged. Surprised? - No. That's just how things usually work. In many cases system isn't being used as it's designed, and when there are such changes, some of the other features break simultaneously. In this case they claimed that the 'browsing mode' was only ment for system administrators. But for some reason it was enable for other personnel too. - Business as usual. There's nothing surprising with that. It's all the time that things like this happen. System is designed for case A but then there's some kind of need which requires configuration changes and then those are made as cheaply and quickly as possible. Which usually means that all the security controls and other 'what if' cases are purely forgotten. Because now it 'works' as they wanted it to work. And they can do what needs to be done.
  

• I'm just wondering why the NFC credit card with loyalty features needs to be read twice when paying, even if the loyalty information has been registered earlier in the transaction. That's about it, I wonder what kind of engineers write this software. A) I register my loyalty identity using the card. B) Then it comes time to pay. I show the card, it says, loyalty information registered. C) Then I'll show the card again, and now the payment is accepted. Why why why, those fail guys got the step B? It's just repetition of the step A. It doesn't make any sense, and makes the the process suck. - Thank you engineers, once again.
• I asked about Telegram chat history retention policy. It's just as it says on their page, but it's written in bit confusing style. So to make it clear. Telegram chat history is kept forever on Telegram servers / cloud / database. Unless, all parties of the chat delete it. Which basically means that most of chats are kept forever. And that's good to remember. If you delete something from chat, you'll only remove it from your personal view. It's still maintained in cloud for everyone else. Just as they say, you can send stuff to cloud and 'delete it', but it actually doesn't mean anything at all. When it's out there, it's there forever. kw: Telegram, chat, message, history, data, retention, deletion, removal, policy, IM, instant messaging, privacy, security.
  
• To continue about crappy code. In one project there's integration (not written by me) which has option to write output as UTF-16 or CP1252. If you enable the option to write out CP1252 files, the data export gets just much slower. I wondered what the option was actually doing. It was pretty clear. If cp1252 option is enabled, add '.tmp' to export filename. Then run the export code as usual. After that open the temp file and read it into memory as one large string. Then remove every second byte from the string and then write it back to disk. - That's marvellous. First of all, it doesn't give a damn about character encodings actually. It'll basically work with < 128 ASCII by design. As well as it's phenomenally slow, because it modified the string in memory over and over again. - But as being said, it still works. And for exports which aren't too big, the time is only a few minutes, so it doesn't matter either when it's run in nighly batch run. Why bother doing things in a complex way, when a simple way works? Ehh... This code has been in production for over 10 years and works well. So there's nothing to complain about it actually. Maybe just a few core weeks or months wasted, but nobody really cares about that. I'm not sure if this works in category, acceptably bad, good enough. But based on the fact that it has been used for so long, I think it does. Yes, it could be in some cases a problem, but in this case it wasn't. So what's the problem? I've seen similar implementations of \r\n to \n conversion. Horrible, but it just works.
• Microsoft kills Skype for Linux no more P2P the cloud will be only option. So much about 'distributed' or 'P2P' IM systems. They want full control.
• Checked out Tosibox - It's a box designed to provide secure network for systems which require security. Yet when checking the list, the first question is that if some of the systems should be connected to the Internet at all. That's the question which has been asked in many cases when something bad has happened. Their use cases list businesses like: Water & Wastewater, Security, Robotics, Lightning, Industry Automation & AMchiner, Home Automation, Food & Beverage, Energy Sector and Building Automation. Ahem, some of these sectors are guaranteed to gather interest from extremely competent attackers. Just makes me wonder. Nothing, nothing at all against Tosibox, their products seem great and well. It's much better that there's a proper attempt to secure systems. But the question remains if it's enough. Based on what we've seen in other cases, the answer most likely is, no, it doesn't actually help. But it makes things bit harder for attackers.
  

• Read this white paper of CPU cache eviction based side channel attack, which allows stealing crypoto keys from other VMs running on same hardware. More compact article about topic at threatpost. http://threatpost.com/en_us/blogs/side-channel-attack-steals-crypto-key-co-located-virtual-machines-110512
• Read this article about Chip and pin (EMV) card copying: Article 1, Article 2
• I have proceeded with PyClockPro project quite nicely. First version was written while reading the specification and trying to figure out the details. So code ended up to be experimental entangled mess. Which was naturally expected from the beginning. Now I'll rewrite it completely so it's releasable. There isn't much code about 12 kilos where about 50% of code is comments, but it's very rich in detail. Original test code was based on three lists, where each list contains entires waiting for next hand. There was list cold, list hot and list test. Basically it's really simple, but when you add adaptivity to the picture it's really easy to get off by one errors when allocation is modified simultaneously when you run the eviciton code. Suddenly cache is just one entry smaller or larger than it was supposed to be. Well, not to bad in reality, but it's not perfect. I really can't accept that. I also have benchmarked and profiled all key sections and design choices. It was really clear that even if many people recommend using linked list with CLOCK-Pro it's really bad idea with Python, it's just simply slow. Native Python list is quick to access with indexes, it's even quicker to access than dictionary with keys. "Walking the list" with for is even faster than that. If you compare it to walking recursive linked list, it's super fast. I now use this one list for meta data and one dictionary for data and reference bits. It seems to be optimal design as far as I know. As soon as code is relesed, I really hope to get feedback, I'm very sure, it's not perfect yet. I also have a few questions about the internal logic, that specification or any other documentation which I have read doesn't really answer. I'll release those questions with first source. I think biggest questions are if hand hot should be allowed to delete all test pages (non-resident cold pages) and if hand hot is allowed to "dive in" the cold page pool in front of hand cold when memory allocation for hot pages is really low. I personally would prefer to skip deleting test pages and instead of looking for hot pages in cold list, I want to expire cold pages instead of hot pages, even if this leads to situation where we have more hot pages than we should have. Main problem with hand hot moving "too far" forward is that actaully hand hot is the primary hand which coordinates adding new entries in clock. Therefore it's important that it really can't move too far forward and something else should be done instead. I have run multiple test runs with vistual clock operation debug output, and it's very clear that if clock works without some of these minor modifications results aren't going to be optimal.  
• I have also studied Sequential Prefetching in Adaptive Replacement Cache (SARC). Unfortunately it's yet another nice algorithm which is patented by IBM and written another experimental code based on specificiation. SARC is really novel solution which allows using same storage space for random read caching, as well as prefetching data for sequential reads to cache, minimizing storage latency to applications under different mixed workloads. I just wonder if I could rip parts from SARC in incorporate those to CLOCK-Pro. Like the prefetching detection counter system. I already have implemented mixed random read, and writeback modes when using same shared memory. Adding prefetching to that, could be really interesting idea. That's definitely something to think about.
  
• Finished reading Proportional Rate Reduction for TCP (TCP PRR) specification. Make fast retransmit more reliable and reduce retransmit bursts. 
• Had a very long discussion with colleagues if it's better to have wider and shorter roles or narrower and longer roles in projects. I personally think that having short roles where you only do one step in project might be harmful. You can't really even learn from mistakes what you made, if you aren't ever informed about those mistakes. When you have longer role (in time), it means that you're part of project from the beginning to the very end. If any decisions made at the beginning turn out to be bad, you know exactly why those were bad and you can avoid those in future. This approach also makes people to think in detail how things should be done. If I know that someone else is going to operate the system in production which I'm writing, I can make it really crappy. It's not my problem. It got delivered and accepted. But if I know that I'm responsible for the operation of the whole project, I'll make it so that it's easy to maintain and preferably works very reliably so I'm not getting bothered by random problems which might arise from poorly designed / implemented project. But this is just my personal opinion, as well as everything  else in this blog. This is also linked to DevOps role, because if designers make bad design, or devs make poor code, it's still after all operators problem. But if you're DevOps, then it's your own problem.
• Read even more stuff about OpenStack. Maybe I should install open stack test environment just for fun. It's great if there would be nice cloud services on higher than IaaS level which do not cause vendor lockin.
• Checked out a few tools for documenting business processes. Also refreshed my memory by reading about business process management, business process modeling, continuous improvement process and total quality management. Nothing new, but it's good to remind your self about somethings from time to time. 
• Read this great post about MongoDB gotchas, which new deveopers encounter with MongoDB. Nothing new in this article either. 
• Encountered (from a friend) an interesting PHP malware, which was obfuscated using recursive encoding eval(gzinflate(base64_decode("Payload"))). Guy who had that script said that it was encrypted. Duh! That's not an encryption! It's funny to call gzipped and base64 erncoded data to be encrypted. I wrote small Python script and after about 50 decoding iterations I got the source out. It was some kind of proxy tool which they can use to scan, attack and exploit other systems. As well as carry data back to source. File was dropped from Germany server hosting and inside script there was IP addresses referring to another German hosting site. I'm sure those servers were hacked too. Unfortunately there's no reply to abuse report from either hosting company yet.
• Finally just to cheer you up, I would like to announce that all content of this site is produced using recycled bits.

• Multi-tenant, cloud based, on demand capacity scaling. Average server load is under 10%, so why do we need the 90% for?
  
• Intel HTML5 tools
• SQLite3 parallel access
• Error detection, handling and self recovery
• BGP / Anycast
• Implementing message queues in relational databases, I think I've covered this topic already. Not optimal, but works.
• Database as queue anti pattern, and again. I even posted longer post about this. But other queue solutions might not be much better, if data is being queued for too long.
  
• I have seen way too many directory / file based or database based queues. - Yet those do work as well. My current mail server is actually doing just that. (Postfix, with maildir)
• So about using database as queue, here are a few problems and few suggestions how to improve it (a post by me). 
• How to design rest APIs for Mobile.
• Web store, card acquirer, point of sale, bookkeeping, invoicing, ERP, card issuer, card transaction clearing, order number, data structure, ticket sales, reference numbers.
• Physical Aggregation, Fabric Integration, Subsystem Aggregation. Solution where there are several completely separated modular subsystems like processing, memory and I/O.
• Whole rack is "single computer" with shared resources. (Rack-Scale Architecture, Open Compute Project, silicon photonics technology, Intel Atom S1200)
• Slow database server? Yeah. After checking why it's slow. I found out process which repeatedly requested 69421 rows 536 times in 10 minutes. And another query by another process did request 55512 rows 536 in 10 minutes. So it seems that some of these queries were basically running in never ending loop. This started interesting question, should the code be fixed, is it designed correctly, why data is being requested all the time. How much memory server should have, does the requests even hit indexes and so on. Engineers though it would be best to add a lot of memory and CPU resources to the system, because it's now slow. Finding out these things took less than hour, fixing the issue. It remains to be seen if it ever happens. I guess the fix is adding more resources after all.
• Performance issues, endless loop without delay. Why solutions x,y,z and being applied when nobody knows what the problem is? I have seen this happening over and over again.
  
• Discussed above matters with independent IT professionals and they all agreed that these problems are really common. Code is horrible, there's a lot of lock contention and so on, performance tanks, even adding resources might not help. When things are done so badly. One guys said that they're having huge problems with their ERP systems database. The ERP system alone runs just fine, but there are some reports which are written by 3rd party consults and those seriously abuse locking causing lock contenting and basically stalling whole database server.
  It's generic problem that people do not understand the problem is nor they are able to use tools which would reveal what the problem is. Instead of doing that, they just do something random and hope it would fix the problem.
  
• Total home renovation project, is still taxing my IT 'hobby'. I've been so busy with it. I'M getting new just about everything. Every surface in the apartment is being renewed as well as plumbing, networking, electric cabling, floors, walls, kitchen, home appliances etc. I hope this means that I don't need to worry about these things in next 20 years or so.
• Haiku OS - Played with it for a few hours.
• Tachyon - Fault Tolerant Distributed File System with 300 Times Higher Throughput than HDFS
• Designing REST JSON APIs - Nothing new at all. But could be problematic with mobile apps, if round trip times are high. All required data should be received using one request, even if it technically involves multiple resources. So some kind of 'view' resources should also be available which technically merge multiple lower level resources.
• Working with systems which are broken by design, is really painful and frustrating. There are multiple scheduled tasks, which might not run or can run, if any of those tasks does not work, data is is lost and requires complex manual recovery etc. I don't want to see that kind of systems at all anymore. It's so darn painful doing the manual recovery process, especially if it rarely happens. Let's say the crap fails 0,5-2 times / year. Nobody even remembers how the systems should get recovered. Simple and reliable, are the keywords which I really like. Sometimes I do prefer multi-step process, where in each step you can check the data and state easily. So you'll know that everything is good so far. One big black box which gets something in and spits out something, might be a lot harder to debug.
  
• More great code. Let's assume we have huge database. Each database row got flag if the row has been processed. I would prefer pointer to monotonically increasing counter, instead of per row flag. But his is someone implemented it. Now they're doing it like this.
  

  SELECT type, and_so_on WHERE processed = 0
  
  The data is getting processed:
  
  if type == do_stuff_type:
      do_stuff()
      update processed = 1
  
  Well, guess what? All rows which aren't typed with do_stuff_type are selected over and over again. How about just updating the processed flag outside the if statement? Or maybe skipping the records on select query (which is still inefficient). Current implementation is exactly what I didn't want to see. Sigh.

• Configured mail client to show spamgourmet and trashmail specific headers by default, so I don't need to use show all headers or message source to see a few important fields.
• PGStorm accelerating PostgreSQL with GPU. 
• MongoDB hash based sharding
• Wildfire.py - Self-modifying Python bytecode: w.i.l.d.f.i.r.e
• Hardware based full disk encryption (FDE)
• Firefox seems to use SQLite3 in WAL mode, that's good choice when there are lot of writes.
• What's new in Linux 3.9
• Experienced developer can give you huge list what to do and why, and what not to do and yet again exactly why. I'm seen millions of ways writing extremely bad and unreliable code. I can tell exactly why not to write such code. Nothing kills more productivity than totally unreliable code.
  Use transactions, locking, handle exceptions, give clear error indicaton, log possible issues. Use auto-recovery, if possible. It's not that hard, it should be really obvious for anyone. Don't write stuff that totally kills performance, use batching, sane SQL queries, indexes, etc.
• WiDrop - Wireless Dead Drop. As far as I know there isn't one in Helsinki yet. Maybe I should build one? Just have to ask some friends living in really center of town to run it. Or maybe I could find some local business to host it. Which probably won't work out after someone abuses the dead drop with wrong stuff.
• When using Windows as file server, I just wonder why people always also give remote desktop access to server. It's just like when people share sftp / ftp / ftps / scp accounts, they usually also give always shell access. I guess that tells something about the administrators.
• SaaS company providing superior Web-based security solutions for businesses, institutions, and government agencies to securely encrypt, time stamp, store, transmit, share, and manage digital data of any kind and size across a broad array of operating systems and devices, ranging from smart phones to Supercomputers. These services are further supported by our comprehensive and certifiable audit trail, including irrefutable time stamping. The proprietary methodology at the core of safely locked allows for a wide range of applicability of its software and services, all of which brings to the world market innovative, highly secure, interoperable and cost-effective services and products. - Sounds pretty cool.
  
• Inception - Inception is a physical memory manipulation and hacking tool exploiting PCI-based DMA. The tool can attack over FireWire, Thunderbolt, ExpressCard, PC Card and any other PCI/PCIe interfaces. - If there's physical access to your systems, you're so owned.
• Linode started to use TOTP
• Flash Cache, Bcache, Bcache Design
• WiDropChan anonymous wireless wlan chan with HTML5 off-line support. Just a thought play. Could be useful for someone. Providing pseudonymous access with private messaging & attachments would be nice bonus. Server software would provide basic access control, flooding protection, captive portal and HTML5 client off-line features. To update data it would be enought to visit the page when you pass by. After that, you could handle messages and files offline.
• PCBoard BBS system - Sorry no time to write about it. I would have been just a good example about file based locking and multi-computer shared environment.
• Babel routing protocol
• Thinking about games and their closed world and money systems is great excersice. How do you make the game fair? How do you prevent inflation, deflation and generally control the supply of money in different economic situations. Making game fair in a such sense that all money doesn't come to one user etc. I did spend a lot of time thinking about these concepts, but I didn't ever write about it nor I did write any code. 
• Big-O Cheat Sheet - This webpage covers the space and time Big-O complexities of common algorithms used in Computer Science.
• When bad data is received, what should be done? Halt, continue, log error, send alert? I often have found out that the best way is to halt, it forces the problem to be investigated. If processing is continued and error is only logged, it might take months before anyone notices that things aren't as those are supposed to be.
• Had a long discussion with a good friend about: fluid intelligence, crystallized intelligence (aka wisdom)
• Read: The Dangers of Surveillance - Neil M. Richards - Washington University School of Law
  
• Strangest problem ever. I made too large transaction by giving command:
  delete from junktable;
  commit;
  It took a long time and after a the commit log got huge the database service crashed. After that, when restarting database server, the crash recovery crashed. I were forced to delete the transaction log. After all this the database was totally corrupt. Great going! Luckily I was using my test environment and not a production environment. - Phew!
• I really dislike programs which malfunction so that they require operator / admin attention. When things are out of beta, things should just work, work and work. Not fail randomly giving headache for everyone.
  
• Bitmessage load spikes on higher tier streams - Forever continuing retransmissions. Protocol used to synchronize data between nodes in same group not described. Possible traffic timing correlation attacks when using out bound message relay. Passive mode makes retransmission and work proof even worse.
• Skimmed checklist manifesto, and watched Fukushima Two Years After documentary, where they analyzed the failures in essential processes of cooling the reactors. In very short from, after power outage the passive emergency cooling system was off and they didn't even realize it.
• Just like database managers saying, yeah, there is that commit log, just delete it. Great choise guys, it seems that you don't know what the function of journal is after crash. It's true that recovery after crash is faster if journal is deleted. But it leads to data corruption. Great move, but seems to be one of the tools in DBAs standard toolset.
  
• Spanning tree, real-time web applications
• Tested: Freedcamp, Trello, Asana, Apptivo - For software development team work I liked freedcamp, for very simple task management I loved Trello. But Apptivo seems to be the most complete product of these. Maybe bit too heavy for simple tasks, but in general seems to be the best too of all of these.
• Bitmessage - Only route refresh messages should be flooded. Flooding every message to every stream node is very bad tihng. I just remember too well how much fail original Gnutella protocol was. It did work well with a few nodes, but it was totally unscalable by design. Message types (inventory, getdata, senddata, sendpeers)
  Client allows system to be flooded with stale TCP connections. Because I can complete 3 way handshake using raw sockets, I could pretty easily flood all nodes on the network with countless stale connections. I didn't try what would actually happen if I would flood all network nodes with millions of stale connections. But it seems that there's a real problem there. First of all tons of connections shouldn't be allowed from same it, as well as there should be some limits which would prevent keeping stale connections alive for as shorter time. Now connections remain open for quite a long time (too long?), allowing attack to be effective without any additional handshake or state.
  A) Do not allow tons of connections from same IP, especially if there hasn't been ANY negotiation. This flaw makes attack super trivial and easy. I can disable ability to connect new nodes for whole network from one of my servers with gigabit connection in a few minutes and that's super trivial and easy to do.
  B) Because client doesn't recover from the initial attack, there's coding flaw. Normally client should return to 'normal state' after those connections die for what so ever reason. But this doesn't seem to be happening. Client can't form even outbound going connections after attack.
  All this is followed by the classic: socket.error [Errno 24] Too many open files.
  So there are at least two vulnerabilities. Third one is pretty bad too, they allow fake peer information to get propagated throughout the network. This can be also utilized to attack other TCP services by making almost all Bitmessage clients to connect those. There's no check if peer is valid before propagation.
  
• I had nine computer's running Edonkey2000 cluster, which harvested files from Sharereactor. Including my own control software which took care of load balancing and allocating files based on disk space and bandwidth demand & peers on different servers.
  
• Early ed2k server overloadaded when there were too many peers connecting to it. Worst part of it that it only send list of N peers to every client. So when swarm or client group downloading the same file grow large enough, all new peers got only information about those limited number of peers connected to the server. Rest of the peers were unconnected because there was no peer to peer gossip protocol with early servers.
  
• I convinced the ED2K guys to select rarest vs random block when selecting which block should be downloaded from peer which got multiple required block available. I also told the GNUnet developers that the peers would need local block cache eviction. Old versions didn't evict blocks and when cache got full, those were simply unable to insert new blocks. Of course this isn't a problem as long as there's huge churn on peers, which install client run it for a while and forget it. But this totally ruins help provided by peers which are connected all the time and would be able to provide considerable resources to the network, if the cache would be fully utilized. Note! It's not cache, it's data storage, which remains persisted on disk when client isn't running.
  [2015 note] This is exactly what I would also like to see from Tribler.
  
• Read: Introduction to financial economics by Professor Hannu Kahra
• Managed Wlan systems, and configured one for a customer with five base stations
• MitMproxy - Yet another tool for hijacking connections and stealing data
• Finished reading Secrets of Big Data Revolution, by Jason Kolb and Jeremy Kolb
• Watched Google I/O 2013 Keynote
• Studied: software defined storage, universal storage platform, virtual storage platform, system storage san volume controller. storage hypervisor, virtual software, Geo fencing, location intelligence, Integration architect
• User interface design, general usability (UX), and viewing the product usability from end users and customers roles.
  
• General Data Protection Regulation
• UnQLite - An Embeddable NoSQL Database Engine
• Refreshed my memory about Freemail (email over Freenet) documentation and how they check if certain keys are being used and which keys should be used to publish new messages
  
• Just keywords: Predictive modelling, Intermessage delay, asm.js, Gitian, yappi, pump.io
  
• Bitmessage test attack client goals: Network propagated, persistent attack (invalid message getting propagated by the network, but still crashes the peer after it when being processed by the client). As example, packet passes networking and data store parts, but the user interface displaying notice about it crashes the client. In some cases this would even cause the client to crash again when it's restarted. Only purging the data store would fix this issue.
• I like web apps because those work on all platforms: Sailfish, Firefox OS, Tizen, Ubuntu, HTML5, COS.
  
• What's the point of using state of art VPN software, when it's configured not to use keys and passwords being used with it are lightly to say simply moronic. It's just matter of time before the state of art security is broken.
• SQLite3 3.7.17 release notes
• SQLite3 Memory-Mapped I/O
• Bitmessage Fail -  Number of connected hosts isn't managed properly, can easily lead to several problem and running out of TCP sockets. As well as inventory message flood allows memory consumption attack (which caused remote clients to crash and therefore lead to situation where I detected connection management issue) - Quite good reasons NOT TO run P2P software on systems doing anything else than running just the P2P system itself. Always properly isolate P2P systems from all other systems. Bitmessage also failed to exclude 172.16.0.0/12 addresses. So it was possible to make the Bitmessage clients to connect local address space by spoofing peer addresses like described above.
  Bitmessage vanity address generation is trivial. All it takes, is changing one line in source code and some time.
  Bitmessage potential timing attack... You can flood or crash connected nodes and see how quickly the message received confirmation message comes. Also timing attacks might reveal who's the sender in case of distributed lists. Just connect really many nodes, do not relay the message your self and observe order which the message is getting offered to my node from other nodes. Restructure network, crash a few peers and recheck. Address messages can be also be used to check out the network structure, because messages are flood casted. Send message to only one peer and see which order and after what time other peers offer it back to you.
  
• Native applications versus HTML5 applications (in Finnish)
• HTML5 features you need to know
• Non-blocking transactional atomicity
  
• Good reminder: You're dangerously bad at cryptography - Yes, it's hard as any other security related topic. Getting some basics right is quite easy, but after that getting it absolutely right is nearly impossible.
• OpenVPN, SSH and Tor port forwarding & tunneling
• Unhosted applications - Freedom from monopoly - Distributed Standalone Applications using Web Technology
• Parallelism and concurrency need different tools
• Checked out Wise.io
• Is NUMA good or bad? Google finds it in some situations up to 20% slower. - I guess it's all about memory access patterns.
  
• In the upcoming Google App Engine 1.8.1 release, the Datastore default auto ID policy in production will switch to scattered IDs to improve performance. This change will take effect for all versions of your app uploaded with the 1.8.1 SDK. - Ok, so they didn't like the original concept where 1000 consequent keys were allocated at once for a peer. Even that didn't guarantee incremental key allocation, because each thread got it's own key space for performance reasons.
• Studied more Security Information and Event Management (SIEM)
• I'm one of the StartMail beta testers.
• Started to use Comodo Personal Email Certificate as optional to my GnuPG / PGP keys.

Link: 2013 mega dump 2 of 2

To celebrate the beginning of year 2013 I have this little assorted post. This is not even nerly everything. I could have written several posts about each topic covered in this post. I'm just sorry I don't have time for that. This is just a web log of stuff I do, I really can't cover all the details.

• Finished reading Innovator's Dilemma. I can see many similarities in the business I'm currently in. How things change, and how everything new is impossible or hard, things should be done as those are always been done. Well, maybe it's time to change things, even for better?
  
• Veikkaus.fi web site is finaly allowing longer passwords up to 32 chars including special characters. Earlier their policiy did only allow 8 chars including a-z, A-Z, 0-9, which is bit outdated policy afaik.
• Reminded my self about HTTP header fields (RFC2616). Mean while in process, found out that really many sites give out conflicting information in Cache-Control header. Private means that data should be cached, but only for the user. While no-cache means that data shouldn't be cached. So why site then defines private and no-cache simultaneously? Also some browsers require usage of must-revalidate. Why? If there's no-cache filed given, why shoud cached data be revalidated when there is no cache in first place? Of course we have Expires field and old Pragma and all tricks involved with those. But basically Cache-Control: no-cache should be enough and Expires field can be set to 0 or point to passed point in time, but it's not required.
  
• Did read huge long discussion about CloudFlare. Point of discussion was if CloudFlare is making your site faster or not? Well, basically cloudflare works for caching proxy for resources which are cacheable, but for non-cacheable resources, it just adds delay. Therefore it's really important to be smart about what is being cacheable and what's not and especially not to declare resources that are cacheable to be non-cachable like Google does.
  
• Worked hard with one minor project to get it off the blacklists. Let's say that the abuse control policy wasn't best possible. Domain got blacklisted and now I had to fix first the app and then get it removed from the lists. It can be very tedious work.
• Limited number of parallel (concurrent) HTTP sessions in my browser to improve performance. I'm now living in rented flat and using ridiculous (compared to my normal fiber connection) 3G connection. After limiting number of parallel connections everything seem to be working much better. Because there aren't too many TCP connections competing from this bandwidth. I also might tune down my TCP-stack settings which are absolutely maxed out for fiber connection. (Like initial TCP RWIN 64k etc.)
• Read a few posts wondering when IPv6 will really kick-off? Here's one. Well, I have to say that I fully working native IPv6 connectivity, and I'm happy with it. I'm really not using NAT, it seems that many people do not understand network interfaces at all. They also do not understand benefits binding services to only selected interfaces.
  
• Just found out that my 3G internet provider doesn't block outgoing TCP port 25 at all. That's quite interesting, I assumed that all consumer grade network connections block port 25 access. because even many business network connections blick it, unless you make it clear to the provider that you don't want it to be blocked.
• Encountered SQL gotcha when wondering why one app didn't behave like it should have. It seems that SELECT key,value FROM data WHERE value!=0: leads to interesting results. It returns only fields where key is non-integer.  So from 0 1 1.1 2 2.1 3 3.3 only values 1.1, 2.2 and 3.3 get selected. If I want it to return all, like non-zero values, it requires adding decimal to the query. So let's put 0.0 instead 0 there, and then it behaves as expected. It's always good to know your systems thoroughly otherwise you'll find your self having these kind of strange problems.
  
• Reminded my self about WiFi (WLAN) stuff: WDS, roaming (controlled by client), SSID, frequency allocation, encryption modes, etc. Especially for situations when you have larger set of base stations. One of main points was not to use WDS unless it's absolutely required, it'll just make your network slow or superslow in case of several base stations.
  
• We all did read news about fake google.com SSL certificates. As I said, users should first authenticate the site, before users authenticate to the site. There's no reason to trust broken SSL at all.
  
• Read interesting article about Tux3 FS. They sure have some interesting ways of solving things out. Reminds me about database transaction journaling and SQLite3 WAL mode.
• Wondered SLIC BIOS, Sysprep and cloning Windows 7 and Windows 8 machines, whic use SLIC BIOS and special Windows licenses. Found one russian hacker app, which allows to read and decrypt all required data from BIOS allowing cloning of those machines and even re-entering the product key. Basically I don't like the basic concept of embedding licensing information in BIOS. This is just like the secure boot which could prevent loading any other than MS(tm) operating system or so.
• Studied litte more in detail, Windows server / workstation Sysprep for cloning.
  
• Updated passwords and email addresses for most of sites. It's quite painfull process. You'll never know which site will accept what kind of new password etc. Some sites allow only short passwords, some sites allow any kind of UTF-8 passwords without any problems etc. It's also hard to find where to change the password in some sites and some sites simply do not properly confirm if new passwords was accepted or not. Overall quite crappy user experience. One of the crappiest sites (bank btw), didn't accept password which is non-numeric and longer than 6 numbers. (Duh)
  
• One nice article about bluetooth security. I just wonder what kind of fails there will be with the Near Field Communication (NFC) stuff.
• Read nice article how I run my own DNS. Nothing new really, it was just fun to read it. I just wonder why he's using postfix to forward mailto Gmail. I run my own Postfix because I don't want my mail to be forwarded to Outlook or Gmail or any other huge dataware house.
• HTTP headers from my web-server:
  HTTP/1.1 200 OK
  Accept-Ranges: bytes
  Cache-Control: max-age=86400
  Content-Length: 79318
  Content-Type: image/png
  Date: Mon, 31 Dec 2012 20:35:59 GMT
  Etag: "1ff11-135d6-4bfd1068fed7c"
  Expires: Tue, 01 Jan 2013 20:35:59 GMT
  Last-Modified: Sat, 12 May 2012 06:33:06 GMT
  Server: Apache/2.2.22 (Ubuntu)
  Strict-Transport-Security: max-age=86400
  x-mod-spdy: 0.9.3.3-386
  X-Firefox-Spdy: 3
• Closed a few (small traffic) web forums and replaced those with newly created Google+ Communities.
• NitpickerTool.com finally a proper spellchecker. Maybe I should use it too? Just for lulz, they fail... The image on front page http://nitpickertool.com/resources/img/overview.png sends HTTP response header Cache-Control: no-cache. It's totally crazy and pointless. Usually I don't even notice these things, but I'm now using narrowband 3G instead of 1 Gigabit/s full duplex fiber connection. It's so very easy to spot crappy web-sites from the sites where admins know what they're doing. This is perfect example. (btw. Google Engineers do not know either what they're doing, I'll get back to this topic). - Btw. They now have fixed it.
  
• Reminded myself about current Secure Boot UEFI issues with Free Software, and how that's going to possible affect several Linux distributions, unless SB can be disabled by user.
• Studied and tested mod pagespeed. Even I didn't leave it enabled. 
• Wondered why FB sends B*S* information about users to other users. I'm 99% sure that the user isn't following the pages FB claims her to follow. I also later confirmed my suspicion and I was right. The information which FB sent, was based on my profile, but they claimed it was based on her profile. I think that's just plain deception.
  
• Checked out and played with Ninchat. Didn't like their Flash video chat, but otherwise it's nice web-chat.
  
• Watched Indie Game Movie. When stakes are high, it seems that people can get bit messed with their thoughts. Luckily all businesses covered in that documentary did well after all. 
• Military stuff read whole articles from Wikipedia: NASAMS 2, AIM-120 AMRAAM, MBDA Meteor, Iranian submarine force (Kilo (Russian) & Quadir/Ghadir (Iran) & Yono (North Korea) class), supercavitating torpoedoes (Hoot &
  Shkval) and Iron Dome (Israel) antimissile systems.
  
• Studied Missile Development project control goals. Strict requirements for features that has to be demonstrated before proceeding any further with project. Great way to have strict way for Prooft of Concept.
• Studied pricing of one service provider. I got really upset, because their pricing was really confusing. It didn't cover edge cases at all and interaction of different mix and match solutions were also missing. People who make pricelists should really think out the cases, instead of listing just prices. It's just like doing my daily ERP integrations, I get way too often technical description of some API and then question how much it would cost to do this. Well, it's great, we now know that we can REST, but all the details required to generate the actual payload data are still totally missing. "Hi guys, how much does a car cost?"
  
• Laughed at Jysk Xmas website. They are using Azure but doesn't help at all. They send daily emails asking quiz questions and then users rush to site to answer those questions. Guess what, site is absolutely jammed (over 30 seconds / page load) when those mails arrive. At least they were smart enough not to run this xmas campaing using their mail web servers, because I assume situation would have been even worse in that case. If Google App Engine would have been used, I'm sure it would have performed well. (I'll be posting Google App Engine gotchas post bit later.)
• Other stuff I have read about lately: Big Data, Map Reduce, Hadoop. Well, isn't that just tabulating data? Makes me smile! Check out Tabulating Machine. Overall System Security, Data Erasure Procedures, Enterprise License Management, Startup Tech Companies, Hyper convergence products: Nutanix Complete Cluster, Scale Computing HC^3 (Hyper_Converged Compute Cluster) and Simplivity Omni-Cube. Storage systems: Hitachi Data Systems, NetApp, EMC Cisco Systems, EMC, VMware, VCE Vblock. Storage Networks (SAN) Storage Computing: Nimble Storage, Tintri and Astute Networks. Cognos, Sales, CRM, ERP, Data Analysis, ePOS, POS, mPOS, Point of Sale, Customer Loyalty Systems, Store Chain, Big Box Super Market. BigQuery, Data to Intelligence (D2I)
  "Volume, Variety, Velocity", Open Data, Stream Computing, A/B testing, Continuous Deployment, Software Defined Network (SDN), Open Flow, European Public Sector Information ESPI Platform
  Public Sector Information (PSI), Open Knowledge, PoS CRM sale data analysis on customer level.
  
• I'm mentoring one really young but still promising nerd. He has done all the server stuff I have done (VPS, Linux installations, web servers, dns, mail server (smtps,impas,webmail), starttls with authentication, earlyssh, full disk encryption, SSL / HTTPS), some PHP code etc, and he's only 13 yo. 
• IPTV service TVkaista is under threat in Finland, they're facing charges.
• Needed to checkout interesting linux distribution called Mageia. I think Ubuntu is going to wrong direction, that's also the reason why I gave up Windows years ago.
  
• I visited The Next Web / ArcticStartupMeetup at Teatteri Club, Helsinki (13.12.2012).
• Read article: 14 big trends for 2013: Preemptive health care, Predictive data analytics, Algorithmic censorship and algorithmic transparency, Social coding, Liquid data and Personal data ownership.
• Well one company I have been visiting works just on this "Choice Engines" sector and they're currently having problems with system performance and managing all the data they're collecting from users. Here's nice article about the topic. Question remains, how you can avoid being included in these huge datasets? 
• I were in Egypt, there I learned what it is to have super narrow bandwidth. This current 3G connection is a luxury network connection compared to that. In the very monring it was possible to get 100kbit/s (10kbytes/s data rates), in the afternoon not even half of that. Usually packet loss was so high that it was totally impossible to use any services because timeout did hit faster than requests get processed. This is major dilemma. Many countries with good network connectivity assume that these "slow connections" are some kind of DDoS attacks. I just open sockets keep those reserved without doing anything. But it's not true. Getting SSL (HTTPS) negtion done just might take 30 seconds or more. If server isn't receiving request in 30 seconds, it doesn't mean there is anything wrong with it. On my severs I have set many timeouts relatively low, because I got full 1 Gbit/s connectivity between home and server and less than 1 ms latency. Then it's huge surprise to find out that there is 500 ms round trip lantency and bandwidth is well, bit less than what I have used to. Timeouts and limits are really tricky issue.
  
• Read long and excellent posting about atomic transactions and what could go wrong. It was hilarious, getting things right is much harder than anyone could even imagine before trying it in reality. No, it's not ok if you think only your layer. There are multiple layers involved, disk drives, controllers, drivers, operating system, several layers of caching etc. It's really hard to make sure that atomic transaction is really committed successfully. In some cases it's even impossible, because hardware, file system or operating system could mask it from your app that data isn't actually yet permanently persisted. So you think your system is working? Just run high write transaction loads and then randomly yank the powercord.  Did something get messed up, is the system state still consistent? If not, well, now you have failed fix it.
• Finished reading Blog Hypnosis for beginners book... Very important psychological stuff for sales guys. Yes attitude to lower resistance etc. It seems that many Egyptian sales guys follow these base rules.
• Google Sites seem to use no-cache and no-store (as well as noarchive) tags for hosted images. It's absolutely painful to see every image loaded again on every page refresh. I sent them feedback and I'm now really hoping that they will fix this, because the way they're doing it now, its totally crazy. I also keep wondering why they provide Etag for content which is not allowed to be stored or cached. "no-cache, no-store, max-age=0, must-revalidate", ETag:"1355247003982", X-Robots-Tag:noarchive. Google also seems to be very worried about web robots archiving their content. Well, I guess they know how much content for from other sites they cache.
  Set of full headers from my web page, which I weren't too happy about:
  HTTP/1.1 200 OK
  Content-Type: image/png
  X-Robots-Tag: noarchive
  Cache-Control: no-cache, no-store, max-age=0, must-revalidate
  Pragma: no-cache
  Expires: Fri, 01 Jan 1990 00:00:00 GMT
  Date: Wed, 26 Dec 2012 11:09:25 GMT
  Last-Modified: Mon, 09 Aug 2010 16:50:23 GMT
  ETag: "1281372623009"
  Content-Length: 119297
  X-Content-Type-Options: nosniff
  X-XSS-Protection: 1; mode=block
  Server: GSE
  
• Studied features of Linux 3.7 Kernel ARM 64 bit support, nice.  Server side TCP Fast Open, way cool. BtrFS Hole punching is really nice feature. Now you can "resparse" files by deallocating segments of file, when space inside file is freed, without slow and resource consuming copy data to new sparse file. Option to disable Copy-On-Write (COW) is also nice, even if it might weaken data durability a bit (?).
  Supervisor Mode Access Prevention (SMAP), nice. Yet another layer to layered security approach. I just wonder how many layers we actually need. See Intel documentation. JSF also added TRIM support. Many changes to IPv6, NAT and Netfilter. I really hope that nobody want's to use NAT with IPv6.
• Had some light vacation reading, finished: Getting Things Done For Hackers (GTD guide), The Trading Profits of High Frquency Traaders (High Frequency Trading, HFT) - Highly Profitable and Secure business with Sharpe ratio of 9.2. So I had some light vacation reading.
  
• I'm using multiqueue GTD and it works very well. I never "forget" anything. Then it's just another question how to get taxes done from the list, especially the tasks I hate.
• Well long day we left to Cairo 4:20 and got back to hotel 21:34. Now I have seen the great Pyramids and the Sphinx, while getting sand blasted. I'm been inside of one pyramid too. We ate at the Blue Nile river barge restaurant. Friend of mine bought painting of Scarab (The Beetle) on Papyrus. Street sales guys are incredible pests. I'm not buying anything, I dont't have money and yet they continue bombarding you with offers getting more and more ridiculous. I guess you have to have really dodgy car to drive in Cairo, traffic was quite funny compared to ours. I'm sure insurance companies are happy to provide full CDW cheaply, NOT. Even boarding procedures were completely strange. It just seems that they're not able to organize things efficiently there.
  
• Studied more messaging, authenticatio and encryption(?). Is chaffing and winnowing encryption at all? No, I think it's message authentication scheme, which allows you just to send tons of messages and only the recipient knows which messages are fakes and which one are real ones. So using this method, you can communicate in a way confidentially without using any kind of encryption. See: Null cipher, Steganography (Concealed messaging).
  
• Added to Kindle: The Checklist Manifesto: How to Get Things Right by Atul Gawande, The Signal and the Noise: Why So Many Predictions Fail-but Some Don't by Nate Silver, Automate This: How Algorithms Came to Rule Our World by Christopher Steiner
  
• Played with Snappy compression. It seems to be working quite well with the huge XML data files which I'm dealing with daily. See Google Snappy code page.
• Quicky checked out Blake2 hashing and Blake2*p which is fully parallelized version.
• ROR Sitemap, ROR (Resources of a Resource) is a rapidly growing independant XML format for describing any object of your content in a generic fashion, so any search engine can better understand that content. RORweb.com is the official ROR website. - I really dunno what I would do with this, traditional sitemap or url list is just good enough.
• In Egypt they honestly said 3.75G network, in Finland mobile operators claim DC is 4G even if it isn't.
• Studied lightly CDN networks: CloudFront, ChinaCache, CacheFly, of course we're all familiar with old big ones like Akamai, LimeLight, Level3 and EdgeCast. For some reasons CacheFly seems to be pretty slow often to Finland.
• Studied efficient email server design utilizing: Studied maildir, mbox, procmail, and dovecot caching and datastorage solutions in detail. Btw. Afaik dovecot is doing good job with caching and storage. Sdbox mdbox dbox imap cache, updating automatic caching options / mailbox based on client requirements etc. Afaik this is stuff done right. Utilizing memory cache, cache file, two tier storage, automatic cache optimization. Locking minimization, fsync options, disk io minimization, efficient storage file size (dbox), not too small, not too large etc.
• Checked: Sieve script mail filtering with Dovecot
• Read nice crypto article, 7 codes you'll never ever break. Afaik, it's bad that there are mistakes in Kryptos statue, of course it's also in realworld possible that when something is encoded, it's not getting encoded (encrypted) correctly. I just obfuscated my master key passwords using light paper & pen crypto and I have to say, I had to verify encryption & decrytpion three times / passwd, so I'm sure it's correctly encrypted. I do not ever store plain text passwords anywhere. Level of obfuscation / encryption depends from the security level requirement for that password. Because passwords are random nonsense to begin with, those won't give you easy hints when it's correctly deciphered so encryption doesn't usually need to be very strong. Basic ECB works quite well, even if that's not what I'm using (maybe).
  
• These Egyptian guys at hotel didn't know how to use Sauna properly. They didn't throw water on hot stones at all. The sauna was totally incorrect and unbearable dry hot room, until we fixed a few things.
• A friend of mine made an interesting project using MongoDB. This is yet another temporary email service, but it was mostly studying project. See: my10minutemail.com I just whish he would blog more about the stuff he does. He has really thought about details, how to prevent message loss, how to handle message bounces correctly, how to store data to database etc. I have seen so many services which are in production but they haven't really paid enough focus on technical details. One of these services which fail is boun.cr. Their out going mail server doesn't have proper reverse DNS record. That's a major fail and leads to situation where many receiving mail servers to reject forwarded messages.
  
• Played with QR codes and also studied the encoding method, error correction etc. I have to say that I don't like many things I see, but QR coders are well done. 
• Reminded my self about NUXI problem and differences between small and big endian systems. 
• Checked out release documentation of App Engine 1.7.4: Expanded EU Support, Task Queue statistics, Traffic splitting (between application versions), LogsReader and Logs API, makes analyzing logs a lot easier.
  Expanded Datastore query support, DISTINCT queries. DISTINCT returns distinct set of results. Just like you can do with python by forming set from list. set([1,2,3,2,4,3]) returns {1,2,3,4}, or simply in python shell {1,2,3,2,4,3} returns {1,2,3,4}. Distinct queries are based on index so it's pretty good way to get the results, without distinct feature it would be better to normalize data. Except that data store doesn't support joins, so it might make things actually pretty slow, unless you're caching all normalized fields.
• Just some fun:
  JavaScript:
  0==false   // true
  0===false  // false
  1=="1"     // true
  1==="1"    // false
  Python:
  1==True // True
  1 is True // False
  type(True) // bool
  True is bool // False
  That's all basic stuff, we should all know it.
• Python performance play:
  Each if statement is executed 10 million times.
  if True == True : 1.78s
  if True is True : 1.64s
  if None == None : 1.76s
  if None is None : 1.66s
  if True:        : 0.93s
  if 1:           : 0.93s
  pass only       : 0.93s
  
  I have been doing quite many tests like this for my PCP caching class. I just hope I'll get it released soon. It needs a little final touch and it require just the right mood to dig into details.

• Diceware - I can't believe I haven't written about this topic. Of course this is all old stuff and I've as everyone else has known about this for ages. But just not mentioning is a fail. Anyway. Diceware is one way to generate passwords. I don't personally really like it, because it makes passwords so long. I prefer higher level of entropy and shorter passwords. Yet as mentioned before, I often consider passwords just as pre-shared keys Don't care about the content, as long as it's random enough. Only thing which I think is great with Diceware is the fact that it can actually make entering complex passwords fast on mobile, where there's Swype or similar keyboard in use. Because the words being used are already in dictionary it could be great. Only bad thing is that most of apps actually disable dictionary when entering password. Which basically works against this entry method. Then you have to write really long password without dictionary, which is painful. Even more painful than entering shorter complex password? Also shorter complex passwords can be learned without any problem when being used daily. I'm not providing any examples, because I have my own set of password derivation systems.
  EFF New Wordlists for random Passphrases - "It contains many vulgar words" - Hahah. I wonder why people are so sensitive about passwords. If your totally random password is ub1G sH17 h3!d. What's wrong with that. Trust me, it's totally random. Nothing personal. Some password generators even have rules of filtering out offending passwords. But why? It reduces number of available options and entropy therefore.
  
• Dvorak - Another thing, which everybody should know. I've known and used. But it seems that I haven't blogged about it. I've even used Finnish DAS version of it for a few years. Unfortunately many environments doesn't provide it by default. It would be just so awesome if Windows & Linux & Android would allow to select Finnish (DAS) keyboard when required. But without pre-existing support, it's too annoying to configure it. Even if number of systems I use daily is quite limited. 
• How hard can it be to turn on Mobile Hotspot and join it with a laptop. There are just so freaking hopeless people out there. Sigh. Well, it worked, but it took more than one hour. It seems that WiFi (WLAN) is some kind of higher class of science with requires 10 years of academic studies + 10 years of experience to setup and use.
• Absolutely awesome postmortem from Status Exchange Network. Also gives a good view how trivially easy it is to DDoS a website to it's knees, if it contains absolutely horrible and extremely bad recursive code. It's a good question why this trimming happens and view time and not when the data is being saved? Afaik, it's also a bad choice. Why to do same task several times, if doing it once is enough? - Laughable fail, but that happens. I've often mentioned that many programmers don't have a clue what their code actually does. It just works. This should be one of the classic examples.
• Some neat stuff Windows 10 Anniversary update contains: TCP Fast Open (TFO) for zero RTT TCP connection setup. IETF RFC 7413, Initial Congestion Window 10 (ICW10) by default for faster TCP slow start, TCP Recent ACKnowledgment (RACK) for better loss recovery (experimental IETF draft), Tail Loss Probe (TLP) for better Retransmit Timeout response (experimental IETF draft) and TCP LEDBAT for background connections IETF RFC 6817. - Yet all of those options were preknown to me, many of those weren't actually used. Except TFO. I've often also tweak TCP stack settings for systems which require some tuning. It's neat to hear that those are being used by default and do not require registry or tuning with sysctl on Linux. But as we've seen, it'll probably take a long time before applications and server software supports those features. Except of course some high end projects like web browsers and most common web servers, etc.
  

• Sometimes it takes some provoking to get people to access honey pots. It's unfortunately often assumed that the adversary hasn't prepared for what's happening. But it can happen just so easily, that the hunter becomes the pray and the pray is just luring the hunter into honey pot trap
• Played a while with steganographic social overlay network. Which worked very well. Using three layers: First steganography to layer to hide presence of the message. Next generally well known and trusted encryption. And as third layer, very strong niche encryption. This just in case that if the generally well known encryption fails for any reason, there's even worse layer beneath it. But if the niche encryption fails for some reason, it doesn't matter because there's still the well known and generally trusted encryption layer protecting it. Niche encryption might not be better technically than the mainstream alternative, but also it has received a lot less scrutiny and there aren't as good existing theories and tools for cracking it.
  
• Over 2000 industrial controllers (SCADA) are accessible in Finland publicly from Internet, says CERT.FI. What could go wrong with IoT? If even professionals won't get things right, how would the 'it seems to work' random consumers get these things right? 
• Pro-America propaganda on Social Networks by Sockpuppets? - That's great. I was expecting to see this story on Sputnik News, but now it's The Guardian. Actually there's nothing new. Spreading disinformation and propaganda are age old tradition.
  

• Continued studying and playing with Google Cloud Platform
• Noticed that Namecheap failed to use SSL when handling credit card information. Perfect example how you shouldn't do things.
• Read documentation how NSA tracks Tor users using advanced malware and exploits in client computers. De-anonymizing tor network itself seems to be quite hard even for them. Even if it's well known that Tor is low latency network, and therefore doesn't provide protection against "global internet monitoring adversary" and statistical traffic analysis methods.
• Been having long and interesting discussion with one cloud service provider about different pricing models and how those practically affect customers.
• Configured SpamAssassin for my mail server. I personally don't like email filtration, because there's always possibility of false positives.  But I got sick'n'tired of some ridiculous dietary supplement and stock market scam spams. So now I'm filtering mail, but only very obvious cases are automatically rejected. Of course the sender is also informed about the rejection, so they can contact me with alternate method if they really see that necessary. 
• Fall is coming and it's time to do something actually meaningful. Therefore I just ordered new computer with Intel i7-4770K, Kingston 4x8GB (32GB total) 1600MHz CL9 memory and 500 GB Samsung 840 EVO SSD and Asus Z87-PLUS motherboard. Maybe I should have chosen even faster memory for the CPU, I don't really know. But now I got something to play with during the winter. I'm bit disappointed because it's only 2.7 x as fast as my old computer with trusted Q6600 2.4 GHz CPU. New hardware allows me to run LXC and VirtualBox efficiently, and improve security drastically. I don't now use the host for any web activity. The host OS is now minimal platform for hosting several secure virtual machines, allowing 100% separation of data with different security requirements. As well as allowing me to run 100% state reset, anonymous, instances for private communication. When I'm done with what I just did, I simply shutdown the VM and restart it, and all traces are gone, no information what so ever is saved between individual sessions.
• 
  
• This is quite nice project. It's just like my protoobfs (Simple Protocol Obfuscator) suggestion, but this is way more advanced. 
• "Code-Talker-Tunnel - Code Talker Tunnel is a protocol camouflaging tool, designed to reshape traffic output of any censorship circumvention tool to look like Skype video calls. Our software can be used as a SOCKS proxy and therefore it is extremely easy to use it with different anonymity and censorship resistance tools."
• What a joke, BitTorrent guys release "closed source", NSA proof messanger chat application. Ha ha ha, closed source, NSA proof. Sound's really credible.
• It's good point to notice, that because fingerprints can't be changed, those are more like user id than password. So when you ID your self, you could use RFID crypto chip, PIN and fingerprint. But fingerprint alone can't be used as password. Of course you can use fingerprint + pin when you open doors or devices etc.
• Now it's good time to remind yourself what it takes to make the IT project actually successful. 
  1: Have a competent project manager
  2. Have personnel committed to the project
  3. Have efficient Communicate with enough detail and with right team members
  4. Have a realistic plan, with scope, money / resources, schedule and quality
  5. Have organization management committed to the project as well, otherwise step 2. will likely fail.
  
• Have been enjoying one standard ERP-integration project, where customer has totally lost reality with scope, schedule, quality and cost. Based on my experience these projects usually have totally ridiculous scope, compared to schedule and cost. In those cases quality is of course going to be a problem, because there's no time to test anything. Of course there's a solution, let's just push it into production, right? Project is always "late" and things aren't "working". I have also noticed that in these cases customers often require ridiculously detailed documentation, because they don't have any kind of clue what they're doing anyway. Documentation should be like, that if we now fire our system administrators and hire a few "techies" they should be able to run the show based purely on project the documentation. Ehh, it's not just going to happen. If the documentation is very extensive, who's going to maintain it anyway? I often prefer to write pretty generalized documentation, so it won't be invalid after some minor changes. 
  Often in these situations it turns out that the customers project people do not even understand details related to the project. Therefore using the insanely complex system what they requested will stumble quite quickly after the project is finished. There's simply no-one on their side who's able to properly maintain the complex system.
  It's like if we now donate nuclear power plant to Africa and give them quick education how to run the plan. Do you think that the plant will be properly maintained and run for following 50 years? I don't think so. - Yet again, been there done that. So one important rule, don't design, request and order system which is so complex that is grossly exceeds your organizations skill level. Well, in this case it also turned out that the customer actually was totally unable to design the system, that's why they bought the (custom) app outside. I guess this is quite common issue especially on web-side. Where clueless customers want "Disney like" site for 5000€. After that they think that their cheapest they could find webmaster is able to maintain and further develop the site. Well, now it's said. Death march project.
  Even if this might sound bad, it's not. I'm 100% sure this is familiar stuff for every project / IT guy out there. ;)
  To be honest, this isn't so serious. It's much worse when this kind of project has been going on for years and people have been giving absolutely everything trying to make hopeless project to become a success and then whole project is folded. Luckily I'm not in a such situation. But I know many friends who have been. It's not a good situation to be in. What would you think when you have been working very hard and then you're told that everything you've been doing in last five years is actually meaningless. - Duh!
  (If you, the reader, are the customer, this is my honest professional personal opinion, and that's it.) 
• This Pizza Worm review made me smile. Unfortunately they didn't reach 100 pts with super burp in that review.
• Read the interesting article where they told how it's on NSA's agenda to ruin security of many applications. I think they have been successful at least with IPsec, it's just totally useless and inoperable even with firewalls from same manufacturer.
• All this darkness in Finland is miserable. That's why I just ordered 65W 6500K EFL full spectrum lamp as my wake up light. It's equal to about 400W halogen lamp. Every at 6:00 it turns on and blasts me up from bed with bright light (as Gremlins would say). It's completely true that when it's summer in Finland, it's better to enjoy about it. Because rest of year will be gold and dark. During winter months, it's pitch black when you go to work and when you leave from work. You can see sun during lunch time if you're lucky.