chardet/README.md

82 lines
1.8 KiB
Markdown
Raw Permalink Normal View History

2013-03-03 20:05:01 +00:00
2013-05-08 06:23:48 +00:00
chardet [![Build Status](https://travis-ci.org/runk/node-chardet.png)](https://travis-ci.org/runk/node-chardet)
=====
2013-04-30 13:57:19 +00:00
Chardet is a character detection module for NodeJS written in pure Javascript.
Module is based on ICU project http://site.icu-project.org/, which uses character
occurency analysis to determine the most probable encoding.
## Installation
2013-11-15 05:08:09 +00:00
```
npm i chardet
```
2013-04-30 13:57:19 +00:00
## Usage
To return the encoding with the highest confidence:
2013-11-15 05:08:09 +00:00
```javascript
var chardet = require('chardet');
chardet.detect(Buffer.alloc('hello there!'));
2013-11-15 05:08:09 +00:00
// or
chardet.detectFile('/path/to/file', function(err, encoding) {});
// or
chardet.detectFileSync('/path/to/file');
```
To return the full list of possible encodings:
```javascript
var chardet = require('chardet');
chardet.detectAll(Buffer.alloc('hello there!'));
// or
chardet.detectFileAll('/path/to/file', function(err, encoding) {});
// or
chardet.detectFileAllSync('/path/to/file');
//Returned value is an array of objects sorted by confidence value in decending order
//e.g. [{ confidence: 90, name: 'UTF-8'}, {confidence: 20, name: 'windows-1252', lang: 'fr'}]
```
2017-10-16 00:46:01 +00:00
## Working with large data sets
Sometimes, when data set is huge and you want to optimize performace (in tradeoff of less accuracy),
2017-10-16 00:46:01 +00:00
you can sample only first N bytes of the buffer:
```javascript
chardet.detectFile('/path/to/file', { sampleSize: 32 }, function(err, encoding) {});
```
2013-04-30 13:57:19 +00:00
## Supported Encodings:
* UTF-8
* UTF-16 LE
* UTF-16 BE
* UTF-32 LE
* UTF-32 BE
* ISO-2022-JP
* ISO-2022-KR
* ISO-2022-CN
* Shift-JIS
* Big5
* EUC-JP
* EUC-KR
* GB18030
* ISO-8859-1
* ISO-8859-2
* ISO-8859-5
* ISO-8859-6
* ISO-8859-7
2013-05-04 09:27:28 +00:00
* ISO-8859-8
* ISO-8859-9
* windows-1250
* windows-1251
* windows-1252
* windows-1253
* windows-1254
* windows-1255
* windows-1256
* KOI8-R
2013-11-15 05:08:09 +00:00
Currently only these encodings are supported, more will be added soon.