uncleflo

UTF-8 Sampler

http://kermitproject.org/utf8.html

Saved by uncleflo on June 23rd, 2019.

UTF-8 is an ASCII-preserving encoding method for Unicode (ISO 10646), the Universal Character Set (UCS). The UCS encodes most of the world's writing systems in a single character set, allowing you to mix languages and scripts within a document without needing any tricks for switching character sets. This web page is encoded directly in UTF-8. This, however, is a Web page, which started out as a kind of stress test for UTF-8 support in Web browsers, which was spotty when this page was first created in the 1990s but which has become standard in all modern browsers. The problem now is mainly the fonts and the browser's (or font's) support for the nonzero Unicode planes (as in, e.g., the Braille
and Gothic examples below). And to some extent the rendition of combining sequences, right-to-left rendition (Arabic,
Hebrew), and so on. The subtitle above shows currency symbols of many lands. If they don't appear as blobs, we're off to a good start!

sample test utf8 unicode character category example document reference language sentence method encode universal system allow webpage sequence combine rendition font browser support extent currency preserve development administration

Unicode Character Categories

http://www.fileformat.info/info/unicode/category/index.htm

Saved by uncleflo on December 27th, 2017.

Each unicode character is assigned a category. This is the complete list of categories.

lookup reference development unicode category codepoint code character format letter mark number universal howto pcre regex regular expression information

Unicode Categories

https://www.regular-expressions.info/unicode.html

Saved by uncleflo on December 27th, 2017.

In addition to complications, Unicode also brings new possibilities. One is that each Unicode character belongs to a certain category. You can match a single character belonging to the "letter" category with \p{L}. You can match a single character not belonging to that category with \P{L}. Again, "character" really means "Unicode code point". \p{L} matches a single code point in the category "letter". If your input string is à encoded as U+0061 U+0300, it matches a without the accent. If the input is à encoded as U+00E0, it matches à with the accent. The reason is that both the code points U+0061 (a) and U+00E0 (à) are in the category "letter", while U+0300 is in the category "mark".

regular expression unicode category development codepoint information reference howto guide lookup pcre grep php learn explanation introduction grapheme

How to setup your PHP site to use UTF8

https://allseeing-i.com/How-to-setup-your-PHP-site-to-use-UTF8

Saved by uncleflo on June 27th, 2017.

According to Google (via: DF), UTF8 is now the most popular character set on the web! I wonder how much this is down to sensible defaults in web authoring tools, rather than a conscious shift in mindset. It's a long time since I looked at it, but as far as I can remember Dreamweaver defaults to UTF8 for new web pages, so a lot of beginning web designers are probably building Unicode sites without even realising it. I think there are a couple of reasons that many web designers and developers still aren't using Unicode across the board.

reference technical development administration utf-8 utf8 db database php webpage unicode site advice lookup server guideline practice web code setup character set

The Good, the Bad, and the Ugly: What Happened to Unicode and PHP 6

http://www.slideshare.net/andreizm/the-good-the-bad-and-the-ugly-what-happened-to-unicode-and-php-6

Saved by uncleflo on October 11th, 2016.

n the halcyon days of early 2005, a project was launched to bring long overdue native Unicode and internationalization support to PHP. It was deemed so far reaching and important that PHP needed to have a version bump. After more than 4 years of development, the project (and PHP 6 for now) was shelved. This talk will introduce Unicode and i18n concepts, explain why Web needs Unicode, why PHP needs Unicode, how we tried to solve it (with examples), and what eventually happened. No sordid details will be left uncovered.

php5 php unicode change project difficult history internationalization introduce concept native language charcode

decodeunicode.org . Last Entries

http://www.decodeunicode.org/

Saved by uncleflo on April 4th, 2015.

unicode type entry recurse decode typesetting typography complete symbol update character lookup reference information global

JavaScript has a Unicode problem · Mathias Bynens

https://mathiasbynens.be/notes/javascript-unicode

Saved by uncleflo on April 4th, 2015.

The way JavaScript handles Unicode is… surprising, to say the least. This write-up explains the pain points associated with Unicode in JavaScript, provides solutions for common problems, and explains how the ECMAScript 6 standard improves the situation.

unicode string javascript programming js tutorial bugs website development problem surprise exma situation solution write browser explanation improve database information reference

PHP UTF-8 cheatsheet - blog.loftdigital.com

http://blog.loftdigital.com/blog/php-utf-8-cheatsheet

Saved by uncleflo on January 20th, 2015.

When we started building DropSend, we decided to support all languages worldwide from the start. The interface is currently in English only, but the application can send, store, sort and process your data whatever language you want. As a result, we have a good number of customers out east. To support worldwide languages, you need to use UTF-8 encoding for your web pages, emails and application, rather than ISO 8859-1 or another common western encoding, since these don't support characters used in languages such as Japanese and Chinese.

php programming utf8 unicode digital cheatsheet blog article commercial transparent encode convert issue utf-8 guide information explanation expression render email process language sheet reference

How to support full Unicode in MySQL databases · Mathias Bynens

https://mathiasbynens.be/notes/mysql-utf8mb4

Saved by uncleflo on January 20th, 2015.

For a long time, I was using MySQL’s utf8 charset for databases, tables, and columns, assuming it mapped to the UTF-8 encoding described above. By using utf8, I’d be able to store any symbol I want in my database — or so I thought. While writing about JavaScript’s internal character encoding, I noticed that there was no way to insert the U+1D306 TETRAGRAM FOR CENTRE (팆) symbol into the MySQL database behind this site. The column I was trying to update had the utf8_unicode_ci collation, and the connection charset was set to utf8.

mysql utf8 unicode utf-8 database utf8mb4 encoding internationalization website development administration mathias bynens blog article guide howto information explanation description encode character write support

Handling Unicode Front to Back in a Web App

http://kunststube.net/frontback/

Saved by uncleflo on January 20th, 2015.

Understanding encoding is all fine and good, but there are many gotchas involved in actually building a complex system consisting of many moving parts that uses anything but ASCII characters. This article shows how to get a PHP web application with a MySQL database set up to handle UTF-8 data front to back and explains common pitfalls.

unicode charset utf8 text php encoding encode website application mysql utf-8 blog article concept html translation test display content found administration utility database guide howto development

Unicode Character Classes in ECMAScript Regular Expressions

http://inimino.org/~inimino/blog/javascript_cset

Saved by uncleflo on January 20th, 2015.

ECMAScript, the standardized version of the language JavaScript, defines string values as sequences of UTF-16 code units, not as sequences of characters. This language misfeature complicates Unicode handling considerably. For characters in the Basic Multilingual Plane (BMP) a single UTF-16 code unit (one 16-bit word) suffices. For characters outside this range, two code units are necessary. As an example, the Latin letter A is both one character and one code unit: "A".length === 1, but the Unicode character U+1D400 MATHEMATICAL BOLD CAPITAL A is one character but two code units: "퐀".length === 2. A better language would hide this ugly implementation detail from users, and string attributes such as length would be in terms of characters, not code units. Unfortunately, for historical reasons, ECMAScript forces programmers who want proper Unicode support to deal with raw UTF-16 directly.

javascript regex unicode RegExp text search information blog article guide explanation ecmascript class expression character

Regex Tutorial - Unicode Characters and Properties

http://www.regular-expressions.info/unicode.html

Saved by uncleflo on January 20th, 2015.

Unicode is a character set that aims to define all characters and glyphs from all human languages, living and dead. With more and more software being required to support multiple languages, or even just any language, Unicode has been strongly gaining popularity in recent years. Using different character sets for different languages is simply too cumbersome for programmers and users.

unicode regex regexp characters php mysql sql database encoding encode character language software require pitfall escape property engine flavor requirement tutorial create information

The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!) - Joel on Software

http://www.joelonsoftware.com/articles/Unicode.html

Saved by uncleflo on December 10th, 2014.

Ever wonder about that mysterious Content-Type tag? You know, the one you're supposed to put in HTML and you never quite know what it should be? Did you ever get an email from your friends in Bulgaria with the subject line "???? ?????? ??? ????"?

unicode programming encoding development reference tutorial language software utf8 understand article read guide technical description

Unicode code converter [ishida >> utilities]

http://rishida.net/tools/conversion/

Saved by uncleflo on December 10th, 2014.

Type or paste text in any of the green or grey shaded boxes and click on the button Convert button above it. Alternative representations will appear in all the other boxes. You can then cut & paste the results into your document. Select selects all the text in a box. See the notes at the bottom of the page for other options.

converter tools utf8 conversion online web unicode UTF escaping encode convert utility compare reference demonstration howto library function alternative options administration test

Make your website completely UTF-8 friendly « Adventures in PHP / DHTML / CSS and MySQL

http://pureform.wordpress.com/2008/03/23/make-your-website-completely-utf-8-friendly/

Saved by uncleflo on December 10th, 2014.

Running an Internationalization / Localization [or i18n / L10n] friendly website can be tricky, and sometimes downright maddening for those who haven’t yet delved into the world of Unicode. Allowing your users to post in whichever language and / or characters of their choice to your site is crucial for any modern website.

mysql php utf8 international character set character administration development howto guide configuration website unicode language

UTF-8: The Secret of Character Encoding - HTML Purifier

http://htmlpurifier.org/docs/enduser-utf8.html

Saved by uncleflo on December 10th, 2014.

Character encoding and character sets are not that difficult to understand, but so many people blithely stumble through the worlds of programming without knowing what to actually do about it, or say "Ah, it's a job for those internationalization experts." No, it is not! This document will walk you through determining the encoding of your system and how you should handle this information. It will stay away from excessive discussion on the internals of character encoding. This document is not designed to be read in its entirety: it will slowly introduce concepts that build on each other: you need not get to the bottom to have learned something new. However, I strongly recommend you read all the way to Why UTF-8?, because at least at that point you'd have made a conscious decision not to migrate, which can be a rewarding (but difficult) task.

development encoding utf-8 unicode utf8 html web php character set character international encode document apache mysql administration guide information